# A gauge theory for the 2+1 dimensional incompressible Euler equations

Christopher Eling<EMAIL_ADDRESS>
###### Abstract

We show that in two dimensions the incompressible Euler equations can be re-
expressed in terms of an abelian gauge theory with a Chern-Simons term. The
magnetic field corresponds to fluid vorticity and the electric field is the
product of the vorticity and the gradient of the stream function. This picture
can be extended to active scalar models, including the surface quasi-
geostrophic equation. We examine the theory in the presence of a boundary and
show that the Noether charge algebra is a Kac-Moody algebra. We argue that
this symmetry is associated with the nodal lines of zero magnetic field.

##  I Introduction

In recent years a variety of connections have emerged between the properties
of waves in fluid systems and the topological phases of matter Delplace ;
Souslov ; Tauber1 ; Perrot ; Tauber2 ; Grad ; Liu:2020ksx ; Liu:2020abb ;
Venaille ; Green:2020uye ; Tauber3 . On the fluid dynamics side, the
paradigmatic example is the set of 2+1 dimensional equations known as the
shallow water equations. These equations govern the behavior of a thin layer
of fluid whose height fluctuations are much smaller than its horizontal
extent. For the height $h(x,y,t)$ and horizontal velocity $v^{i}(x,y,t)$,
($i=(x,y)$) the shallow water equations are

$\displaystyle\partial_{t}h+v^{j}\partial_{j}h+h\partial_{j}v^{j}=0$ (1)
$\displaystyle\partial_{t}v^{i}+v^{j}\partial_{j}v^{i}=f\epsilon^{i}_{j}v^{j}+g\partial^{i}h,$
(2)

where $g$ is the gravitational acceleration and $f$ is the Coriolis parameter.
In the 19th century Lord Kelvin showed that the linearized version of these
equations admits waves that are exponentially localized near the coasts. These
waves are chiral modes because they propagate only in one direction around the
coast. For example, for landmasses in the Northern (Southern) hemisphere the
direction of the wave propagation is clockwise (counterclockwise). Later, it
was found that chiral modes are also present at the equator, propagating only
from east to west Matsuno ; Higgins .

Chiral edge modes also famously exist in topological phases of matter,
particularly at the edge of quantum Hall systems Zhang ; Wen . Quantum Hall
fluids are described by an effective 2+1 dimensional gauge field theory with a
Chern-Simons term. For gauge connection $A_{\mu}=(A_{0},A_{i})$ this term has
the form111Here $x^{\mu}=(t,x^{i})$, but note that the expressions in this
notation are not Lorentz covariant as the physics is non-relativistic.

$\displaystyle S_{\rm CS}=\int
d^{3}x~{}\epsilon^{\mu\nu\rho}A_{\mu}\partial_{\nu}A_{\rho}.$ (3)

The Chern-Simons term is invariant under gauge transformations
$A_{\mu}\rightarrow A_{\mu}+\partial_{\mu}\lambda$ up to a total derivative.
In addition, this term is only sensitive to the topology of the manifold
because it is independent of the metric tensor.

In Tong:2022gpg  D. Tong showed that the shallow water equations follow from
the action of a 2+1 dimensional gauge theory with a type of Chern-Simons term.
At the linear level, the theory reduces to Maxwell-Chern-Simons theory, with
edge modes that match the coastal Kelvin waves. Additional work has shown that
other properties of the shallow water system can be expressed in terms of the
gauge theory description Sheikh-Jabbari:2023eba . More recently, it has also
been shown that Tong’s analysis can be extended to include a class of
compressible Euler equations Nastase:2023rou .

A natural question is whether other fluid dynamical systems can be expressed
in terms of a gauge theory. In this paper we investigate whether the
incompressible 2+1 dimensional Euler equation and its “active scalar”
generalizations have a description in terms of a gauge theory. The
incompressible Euler equations have the form

$\displaystyle\partial_{i}v^{i}=0$ (4)
$\displaystyle\partial_{t}v^{i}+v^{j}\partial_{j}v^{i}+\partial^{i}p=0,$ (5)

where $p$ is the fluid pressure and the constant fluid density has been set to
unity. We find that the fluid vorticity $\theta$ and the gradient of the
stream function $\psi$ ($v^{i}=\epsilon^{ij}\partial_{j}\psi$) can be mapped
into the pseudoscalar magnetic field and the electric field vector,
respectively, by identifying the 2+1 dimensional Bianchi identity with the
ideal vorticity equation

$\displaystyle\partial_{t}\theta+\epsilon^{ij}\partial_{j}\psi\partial_{i}\theta=0.$
(6)

We show that a gauge invariant action with an explicit Chern-Simons terms
reproduces the Euler equation (5) and correct relationship between $\psi$ and
$\theta$ involving the 2d Laplacian

$\displaystyle\theta=-\Box\psi.$ (7)

Furthermore, generalizing this action include non-local terms reproduces the
equations of active scalar models, which include the surface quasi-geostrophic
equation Held , where the relation between $\theta$ and $\psi$ is via a
generalized fractional Laplacian operator.

The structure of this paper is as follows. In Section II we review Tong’s
construction of the gauge theory for the shallow water equation. We then use
similar principles to construct a different gauge theory for the
incompressible Euler and active scalar models. In Section III we discuss the
presence of edge modes when our theory contains a boundary. We show that the
algebra of conserved Noether surface charges is a Kac-Moody algebra, which
suggests the presence of conformal symmetry associated with the boundary. We
also argue that nodal lines of zero magnetic field (zero vorticity) are
natural boundary surfaces in flows, and conjecture this explains the
appearance of nodal lines with conformally invariant measures in the turbulent
inverse cascade conformal1 ; conformal2 . We conclude with a discussion of
open questions, including the possibility of extending the construction to 3+1
dimensions.

##  II Constructing the gauge theory action

###  II.1 Shallow water equations

First, we review the construction of the gauge theory action for the shallow
water equations. To construct his action principle, Tong noted that the first
shallow water equation (1) has the form of a conservation law
$\partial_{\mu}J^{\mu}=0$ with conserved current

$\displaystyle J^{0}$ $\displaystyle=h$ $\displaystyle J^{i}$
$\displaystyle=hv^{i}.$ (8)

In a 2+1 dimensional gauge theory, the Bianchi identity implies

$\displaystyle\epsilon^{\mu\nu\rho}\nabla_{\mu}F_{\nu\rho}=\epsilon^{\mu\nu\rho}\partial_{\mu}\partial_{\nu}A_{\rho}=0,$
(9)

which can be rearranged into the form of the conservation of a current

$\displaystyle\partial_{\mu}\left(\epsilon^{\mu\nu\rho}F_{\nu\rho}\right)=0.$
(10)

In terms of the magnetic field $B=\epsilon^{ij}\partial_{i}A_{j}$ and electric
field $E_{i}=\partial_{t}A_{i}-\partial_{i}A_{0}$

$\displaystyle J^{0}$ $\displaystyle=B$ $\displaystyle J^{i}$
$\displaystyle=-\epsilon^{ij}E_{j}.$ (11)

Identifying this topological current with the conserved current of the shallow
water equation yields

$\displaystyle B$ $\displaystyle=h$ (12) $\displaystyle E_{i}$
$\displaystyle=h\epsilon_{ij}v^{j}.$ (13)

The next step is to postulate a Lagrangian in the form of the shallow water
system’s kinetic energy minus its potential energy:

$\displaystyle S_{\rm base}=\int
dtd^{2}x\left(\frac{1}{2}hv^{2}-\frac{1}{2}gh^{2}\right)=\int
dtd^{2}x\left(\frac{E^{2}}{2B}-\frac{1}{2}gB^{2}\right).$ (14)

Varying with respect to $A_{0}$ gives the Gauss law

$\displaystyle\partial_{i}\left(\frac{E^{i}}{B}\right)=\theta=0,$ (15)

where $\theta=\epsilon^{ij}\partial_{i}v_{j}$ is the fluid vorticity.
Variation with respect to $A_{i}$ yields the dynamical equation

$\displaystyle\partial_{t}\left(\frac{E^{i}}{B}\right)+\frac{1}{2}\epsilon^{ij}\partial_{j}\left(\frac{E^{i}}{B}\right)+g\epsilon^{ij}\partial_{j}B=0.$
(16)

In terms of fluid variables

$\displaystyle\partial_{t}v^{i}+\frac{1}{2}\partial^{i}(v^{2})+g\partial^{i}h=0.$
(17)

Clearly, the Lagrangian yields equations of motion that are almost consistent
with the shallow water equations but constrain the fluid’s vorticity to be
zero (the Coriolis parameter $f$ is also neglected here).

In fact, taking the curl of the second shallow water equation yields a second
conserved current associated with the vorticity, with

$\displaystyle\tilde{J}^{0}$ $\displaystyle=\theta+f$
$\displaystyle\tilde{J}^{i}$ $\displaystyle=(\theta+f)v^{i}$ (18)

The idea is to account for non-zero vorticity by coupling the vorticity
current to the gauge connection $A_{\mu}$. One postulates an additional term
in the action

$\displaystyle S_{\rm coupling}=\int
dtd^{2}x\left(\tilde{J}^{0}A_{0}+\tilde{J}^{i}A_{i}\right)=\int
dtd^{2}x~{}\tilde{J}^{\mu}A_{\mu}.$ (19)

This term is invariant under gauge transformations $A_{\mu}\rightarrow
A_{\mu}+\partial_{\mu}\lambda$ because the vorticity current is conserved. The
vorticity current is then expressed in terms of Clebsch scalars $\alpha$ and
$\beta$

$\displaystyle\tilde{J}^{\mu}=\epsilon^{\mu\nu\rho}\partial_{\nu}\beta\partial_{\rho}\alpha.$
(20)

Varying the total action

$\displaystyle S_{\rm total,sw}=\int
dtd^{2}x\left(\frac{E^{2}}{2B}-\frac{1}{2}gB^{2}+fA_{0}-\epsilon^{\mu\nu\rho}A_{\mu}\partial_{\nu}\beta\partial_{\rho}\alpha\right)$
(21)

with respect to $A_{0}$, $A_{i}$, $\alpha$, and $\beta$ yields the shallow
water equations (an additional $fA_{0}$ term must be included to account for
the Coriolis parameter). The final term has a Chern-Simons form if an
additional auxiliary gauge connection $\tilde{A}_{\mu}$ is introduced:

$\displaystyle\tilde{A}_{\mu}=\partial_{\mu}\chi+\beta\partial_{\mu}\alpha$
(22)
$\displaystyle\epsilon^{\mu\nu\rho}A_{\mu}\partial_{\nu}\beta\partial_{\rho}\alpha=\epsilon^{\mu\nu\rho}A_{\mu}\partial_{\nu}\tilde{A}_{\rho}.$
(23)

###  II.2 2d incompressible Euler equation

We now follow a similar type of procedure for the 2+1 dimensional
incompressible Euler equations. To start, we note that the 2d incompressible
Euler equation also has a conserved vorticity current. The ideal vorticity
equation has the form

$\displaystyle\partial_{t}\theta+v^{j}\partial_{j}\theta=0.$ (24)

When this equation is expressed in terms of the stream function $\psi$ we find
Eqn. (6) above. This equation in turn can be expressed as the conservation of
a current with components

$\displaystyle J^{0}$ $\displaystyle=\theta$ $\displaystyle J^{i}$
$\displaystyle=\theta\epsilon^{ij}\partial_{j}\psi.$ (25)

Now we want to match these components to the identically conserved current
associated with the 2+1 dimensional Bianchi identity. Using Eqn. (11) above we
find the following relationship between the fluid variables and the magnetic
and electric fields

$\displaystyle B$ $\displaystyle=\theta$ $\displaystyle E_{i}$
$\displaystyle=-\theta\partial_{i}\psi.$ (26)

The next step is to postulate a gauge theory action. Here we simply use the
first term of Tong’s action

$\displaystyle S_{\rm base}=\int dtd^{2}x\left(\frac{E^{2}}{2B}\right).$ (27)

As we saw earlier, variation of this action with respect to $A_{0}$ gives a
Gauss law. In terms of the Euler equation fluid variables

$\displaystyle\partial_{i}\left(\frac{E^{i}}{B}\right)=-\Box\psi=0.$ (28)

This seems to imply that vorticity is zero as before. However, at this stage
there is no relationship between $\theta$ and $\psi$, they are independent
variables in the action. We can create a relationship by coupling the
conserved vorticity current (i.e., the current from the Bianchi identity) to
the gauge connection of our theory. In this case there is no need to introduce
Clebsch variables and the auxiliary gauge field. The resulting coupling term
is an explicit Chern-Simons term

$\displaystyle S_{\rm coupling}=\frac{1}{2}\int
dtd^{2}x~{}J^{\mu}A_{\mu}=\frac{1}{2}\int
dtd^{2}x~{}\left(BA_{0}-\epsilon^{ij}E_{j}A_{i}\right)=\frac{1}{2}\int
dtd^{2}x~{}\epsilon^{\mu\nu\rho}A_{\mu}\partial_{\nu}A_{\rho}.$ (29)

Varying the action

$\displaystyle S=\int
dtd^{2}x\left(\frac{E^{2}}{2B}-\frac{1}{2}\epsilon^{\mu\nu\rho}A_{\mu}\partial_{\nu}A_{\rho}\right)$
(30)

with respect to $A_{0}$ now yields the correct relationship between $\psi$ and
$\theta$

$\displaystyle\partial_{i}\left(\frac{E^{i}}{B}\right)-B=0~{}~{}\Rightarrow~{}~{}-\Box\psi=\theta.$
(31)

The remaining step is to consider the variation with respect to $A_{i}$. This
gives

$\displaystyle-\partial_{t}\left(\frac{E^{i}}{B}\right)-\epsilon^{ij}\partial_{j}\left(\frac{E^{2}}{2B^{2}}\right)+\epsilon^{ij}E_{j}=0.$
(32)

Multiplying by $\epsilon_{ki}$ yields

$\displaystyle-\partial_{t}\left(\frac{\epsilon_{ki}E^{i}}{B}\right)+\partial_{k}\left(\frac{E^{2}}{2B}\right)-E_{k}=0.$
(33)

Note that

$\displaystyle\frac{\epsilon^{ij}E_{j}}{B}=-\epsilon^{ij}\partial_{j}\psi=-v^{i}.$
(34)

Therefore, the equation becomes

$\displaystyle\partial_{t}v_{k}+\frac{1}{2}\partial_{k}(v^{2})-\theta\epsilon_{kj}v^{j}=0.$
(35)

Using the identity

$\displaystyle\frac{1}{2}\partial_{k}(v^{2})=v^{j}\partial_{j}v^{k}+\theta\epsilon_{kj}v^{j},$
(36)

produces the Euler equation (5) without a pressure term. However, a pressure
term follows simply by including a term $pB$ in the action, where $p$ is an
auxiliary function. This additional term does not change the Gauss law
relation but does add the correct pressure gradient. Thus, we arrive at the
following action for the Euler system

$\displaystyle S_{\rm gauge,Euler}=\int
dtd^{2}x\left(\frac{E^{2}}{2B}-pB-\frac{1}{2}\epsilon^{\mu\nu\rho}A_{\mu}\partial_{\nu}A_{\rho}\right).$
(37)

To summarize, the ideal 2d vorticity equation follows from the Bianchi
identity of the gauge theory. The Gauss law constraint relates the stream
function to the vorticity, and the dynamical equation of motion is the Euler
equation.

Although this is a type of Chern-Simons action, the theory is different from
standard Maxwell-Chern-Simons theory. For example, in Maxwell-Chern-Simons
theory the Maxwell parts (i.e., $E^{2}-B^{2}$) of the action are invariant
(even) under time reversal, while the Chern-Simons term changes sign (odd).
Hence, standard Maxwell-Chern-Simons theory is not time reversal invariant.

In our case, we assume that when $t\rightarrow-t$, $x^{i}\rightarrow x^{i}$,
$A_{0}\rightarrow A_{0}$ and $A_{i}\rightarrow-A_{i}$. Under these
transformations, the electric field $E_{i}$ is even, but the magnetic field is
odd: $B\rightarrow-B$. This is consistent with fluid dynamics, where the
vorticity changes sign under time reversal, but the combination of velocity
and vorticity (i.e., the electric field) is even.

All three terms in the action (37) are odd, and therefore total action is odd
under time reversal. Hence, the resulting classical equations of the motion
are time reversal invariant, as expected. Note however, in a statistical
mechanics picture, the partition function $Z=\int[dA]\exp(-\beta S_{\rm
gauge})$ would apparently change under time reversal.

###  II.3 Active scalar models

The Euler system described above can be generalized to a wider set of “active
scalar” models. The conservation equation has the same form as before

$\displaystyle\partial_{t}\theta+\epsilon^{ij}\partial_{j}\psi\partial_{i}\theta=0.$
(38)

However, the relationship between $\theta$ and the stream function $\psi$
takes the following form in terms of a fractional Laplacian

$\displaystyle-\Box^{m/2}\psi=\theta.$ (39)

Equivalently,

$\displaystyle\psi(\vec{x},t)=\int
d^{2}\vec{y}~{}|\vec{x}-\vec{y}|^{m-2}~{}\theta(\vec{y},t).$ (40)

Different physical models are described by different values of the parameter
$m$. $m=2$ is the Euler system. Other values of physical interest are $m=1$,
which is the surface quasi-geostrophic (SQG) model Held . In this case
$\theta$ is a “potential temperature”. The case $m=-2$ describes large scale
flows of a rotating shallow fluid (known as the Charney-Obhukov-Hasegawa-Mima
model) McWilliams  in the limit of vanishing Rossby radius.

The associated generalized Euler equation has the form Tao

$\displaystyle\partial_{t}(Kv^{i})+v^{j}\partial_{j}(Kv^{i})+(Kv^{j})\partial_{k}v_{j}+\partial^{i}p=0,$
(41)

where the operator $K=\Box^{m/2-1}$. In terms of the electric and magnetic
field identifications (26) the Gauss law and dynamical equation are

$\displaystyle\partial_{i}\Big{(}K\left(\frac{E^{i}}{B}\right)\Big{)}$
$\displaystyle=B$ (42)
$\displaystyle-\partial_{t}\Big{(}K\left(\epsilon_{ki}\frac{E^{i}}{B}\right)\Big{)}+\partial_{k}\Big{(}\frac{E_{i}}{B}~{}K\left(\frac{E^{i}}{B}\right)\Big{)}-E_{k}+\partial_{k}p$
$\displaystyle=0.$ (43)

We conjecture that these equations follow from replacing the first term in
(37) with the nonlocal terms

$\displaystyle
E^{i}~{}K\left(\frac{E_{i}}{2B}\right)+(E^{i}E_{i})~{}K\left(\frac{1}{2B}\right).$
(44)

Furthermore, note that when $m\rightarrow-m$ one can switch the stream
function $\psi$ and $\theta$ in the identifications of the magnetic and
electric fields, but the action itself (37) remains unchanged. For example,
when $m=-2$

$\displaystyle B$ $\displaystyle=\psi$ (45) $\displaystyle E_{i}$
$\displaystyle=-\psi\partial_{i}\theta$ (46)

The Gauss law and Bianchi identity then yield

$\displaystyle-\Box\theta$ $\displaystyle=\psi$ (47)
$\displaystyle\partial_{t}\psi+\epsilon^{ij}\partial_{j}\theta\partial_{i}\psi$
$\displaystyle=0.$ (48)

##  III Edge Modes and Symmetries

The presence of the explicit Chern-Simons term in the action has several
interesting consequences when the theory is considered in the presence of a
boundary. In particular, the interplay of boundary conditions and gauge
invariance lead to the existence of edge modes, which was first discovered in
the context of the quantum Hall effect (see, e.g., Wen:1992vi ; Tong:2016kpv
).

The starting point is the observation that under a gauge transformation
$A_{\mu}\rightarrow A_{\mu}+\partial_{\mu}\lambda$ the Lagrangian (37) is only
invariant up to a total derivative

$\displaystyle\delta_{\lambda}S_{\rm total}=-\frac{1}{2}\int
dtd^{2}x~{}\partial_{\mu}(\lambda J^{\mu}),$ (49)

which for a spatial boundary leads to a extra boundary correction

$\displaystyle\delta_{\lambda}S_{\rm total}=-\frac{1}{2}\int
dx\lambda~{}(J^{\mu}n_{\mu}).$ (50)

Using the fluid identifications (26) this term becomes

$\displaystyle\delta_{\lambda}S_{\rm total}=-\frac{1}{2}\int
dx\lambda(E_{i}t^{i})=-\frac{1}{2}\int dx~{}\lambda(v^{i}n_{i}),$ (51)

for tangent vector $t^{i}$ and normal $n^{i}$. To fix this problem, one
approach is to restrict the gauge parameter $\lambda$ to vanish on the
boundary. However, in fluid dynamics a natural choice of boundary conditions
is to restrict the flow to be purely tangential to the boundary, in which case
the anomalous boundary term is zero.

This is not the end of the story because an action principle only correctly
produces the equations of motion when boundary terms are set to zero. For the
variation of (37) the boundary terms are

$\displaystyle\delta S_{\rm bdry}=\int
dtdx\left(-\frac{n_{i}E^{i}}{B}~{}\delta
A_{0}+\frac{1}{2}\epsilon^{ij}n_{i}A_{j}~{}\delta
A_{0}-\frac{E^{2}}{2B^{2}}\epsilon^{ij}n_{i}\delta
A_{j}-p\epsilon^{ij}n_{i}\delta A_{j}-\frac{1}{2}A_{0}\epsilon^{ij}n_{i}\delta
A_{j}\right).$ (52)

These boundary terms can be set to zero if $A_{0}$ and the component of
$A_{i}$ parallel to the boundary are fixed to be constants on the boundary.
Note that this restricts the allowed gauge transformations to the subset of
transformations that preserve these boundary conditions. Typically, local
gauge transformations are redundancies in the description of the theory and
are distinct from global symmetries acting on actual physical degrees of
freedom. However, restricting the gauge freedom allows formerly unphysical
degrees of freedom to become physical. Hence the restricted gauge
transformations preserving the boundary become global symmetries of the
boundary degrees of freedom.

The conserved charges associated with these symmetries are given by the
Noether charges associated with the gauge transformations. Schematically, the
variation of any Lagrangian yields

$\displaystyle\delta L=E_{\mu}\delta\rho^{\mu}+\partial_{\mu}\theta^{\mu},$
(53)

where $E_{\mu}$ are the equations of motion, $\rho^{\mu}$ represents the
fields, and $\theta^{\mu}$ is the symplectic current. Furthermore, the
Lagrangian itself is a total derivative under the transformation $\delta
L=\partial_{\mu}S^{\mu}$. Consequently, the Noether current
$N^{\mu}=S^{\mu}-\theta^{\mu}$ is conserved on-shell when the equations of
motion are imposed

$\displaystyle\partial_{\mu}N^{\mu}\approx 0$ (54)

On-shell $N^{\mu}$ can be expressed as the divergence of a superpotential
2-form $P^{\mu\nu}$, i.e $N^{\mu}\approx\partial_{\nu}P^{\mu\nu}$. Hence, the
Noether charge $Q$ of the theory is a surface charge via

$\displaystyle Q=\int N^{\mu}d\Sigma_{\mu}=\int P^{\mu\nu}d\Sigma_{\mu\nu}.$
(55)

For the theory (37) the resulting Noether current for gauge transformations
($\delta A_{\mu}=\partial_{\mu}\lambda$) is

$\displaystyle N^{0}=$
$\displaystyle-\frac{E^{i}}{B}\partial_{i}\lambda-\frac{1}{2}\epsilon^{ik}A_{i}\partial_{k}\lambda+\frac{1}{2}B\lambda$
$\displaystyle N^{i}=$
$\displaystyle\frac{E^{i}}{B}\partial_{t}\lambda-\frac{1}{2}\epsilon^{ij}A_{j}\partial_{t}\lambda+\frac{E^{2}}{2B^{2}}\epsilon^{ij}\partial_{j}\lambda+p\epsilon^{ij}\partial_{j}\lambda+\frac{1}{2}\epsilon^{ij}A_{0}\partial_{j}\lambda-\frac{1}{2}\epsilon^{ij}E_{j}\lambda.$
(56)

Using these results yields the following set of $\lambda$ dependent Noether
charges

$\displaystyle Q_{\lambda}=\int
dx\lambda\left(\frac{E^{i}}{B}-\frac{1}{2}\epsilon^{ij}A_{j}\right)n_{i}.$
(57)

In terms of fluid variables, and assuming the boundary is closed surface, the
charges have the form of a $\lambda$ dependent local circulation

$\displaystyle
Q_{\lambda}=\oint\lambda\left(v^{i}-\frac{1}{2}A^{i}\right)dx_{i}.$ (58)

A similar result was found in Sheikh-Jabbari:2023eba  for the Maxwell-Chern-
Simons theory describing linearized shallow water waves. The infinite number
of charges indicates the presence of an infinite dimensional symmetry
describing the boundary system. The algebra of these charges follows from the
Poisson bracket

$\displaystyle\\{Q_{\lambda_{1}},Q_{\lambda_{2}}\\}=\delta_{\lambda_{2}}Q_{\lambda_{1}}.$
(59)

One can define a “charge aspect”
$\gamma=\left(v^{i}-\frac{1}{2}A^{i}\right)dx_{i}$. The charge aspect is not
gauge invariant, changing by a total derivative $t^{i}\partial_{i}\lambda$.
Note that the global circulation charge (i.e. $Q_{\lambda}$ for $\lambda=1$)
is gauge invariant for singled valued gauge transformations.

The gauge non-invariance of $\gamma$ leads to a central extension term in the
charge algebra, which is a U(1) affine Kac-Moody algebra in the form

$\displaystyle\\{\gamma(t,x),\gamma(t,x^{\prime})\\}=-\frac{1}{2}\partial_{x}\delta(x-x^{\prime}).$
(60)

The appearance of the Kac-Moody algebra is typically associated with a chiral
boson theory on the boundary Elitzur:1989nr ; Balachandran:1991dw . The
simplest theory of a two-dimensional boson field $\Phi$ can be defined via the
action

$\displaystyle S_{\rm boson}\sim\int
d^{2}x~{}\partial_{\mu}\Phi\partial^{\mu}\Phi.$ (61)

Solutions to this theory (harmonic functions) naturally separate into left and
right movers, or equivalently in the complex plane, holomorphic and
antiholomophic parts, i.e. $\Phi(z,\bar{z})=\phi(z)+\bar{\phi}(\bar{z})$. It
is well known that the holomorphic (chiral) conserved current of the theory
${\cal J}=i\partial\phi$ obeys the U(1) Kac-Moody algebra (ie, a current
algebra). In terms of a mode expansion ${\cal J}=\Sigma_{n}~{}{\cal
J}_{n}z^{-n-1}$

$\displaystyle\\{{\cal J}_{n},{\cal J}_{m}\\}=-\frac{1}{2}m\delta_{m+n,0}$
(62)

Furthermore, the stress tensor $T$ of the theory can be constructed as the
product of two currents $T=-(1/2)(\partial\phi)^{2}$. The algebra associated
with the stress tensor mode expansion is the famous Virasoro algebra
associated with conformally invariant field theories Goddard:1986bp ;
Ginsparg:1988ui .

The appearance of the Kac-Moody algebra indicates that there is an underlying
conformal symmetry associated with boundaries in incompressible fluid flows. A
natural “edge” is the boundary between regions of positive values and negative
values of the magnetic field $B$, represented by nodal lines of $B$. For $m>0$
in active scalar models this corresponds to boundaries where $\theta=0$. For
$m<0$, $\psi=0$. At these boundaries, the electric field also vanishes, but
the fluid velocity defined as the ratio of $E^{i}$ and $B$ is finite. Also,
the Hamiltonian is zero, switching signs between regions of positive and
negative values.

Topologically protected modes appear when time reversal symmetry is broken, as
in the shallow water system with non-zero Coriolis parameter. While the
incompressible Euler equations are time reversal invariant, two-dimensional
turbulence is asymmetric under time reversal. Interestingly, in 2006 it was
shown numerically that zero vorticity nodal isolines in the turbulent 2d
inverse cascade are identical to the curves forming cluster boundaries in the
scaling limit of statistical mechanical systems at equilibrium conformal1 ;
conformal2 . Turbulent flows in 2d can be created by stirring or exerting a
random force on the fluid at some scale $L_{f}$. Kraichnan Kraichnan
discovered that a “double cascade” appears, where the enstrophy created by
stirring is transferred to small scales (direct cascade), while energy is
transferred to large scales (inverse cascade). When a large separation of
scales $L>>L_{f}$ exists, the physics of the inverse cascade is expected to be
independent of the details of the forcing. Moreover, unlike in 3d turbulence,
the effects of viscosity at small scales can be neglected, and some properties
of ideal flows are likely to be relevant.

The nodal curves are expected to random fractal lines in turbulence, but it is
surprising that the measure on the curves is conformal and described by
Schramm-Loewner evolution (SLE) Schramm . SLE curves have been proven to be
the scaling limits of cluster boundaries formed by various statistical
mechanical models defined on a lattice. At criticality and in the continuum
limit, these models are described by different 2d conformal field theories
(for a review, Bauer:2006gr ). Similarly, different fluid models (ie,
different $m$) correspond to different 2d CFT’s Falkovich  even though the
turbulent inverse cascade system is far from equilibrium.

Because the edge states are associated with topology, it is likely that they
are robust in highly turbulent flows at large Reynolds number. In the
turbulent state a statistical description of the flow is required and the Kac-
Moody and Virasoro symmetries of the boundary edges should be associated with
probabilities. We conjecture that the probability that one or more $B=0$
curves is present in an infinitesimal region around a point in the inverse
cascade is conformally invariant and is associated with the stress tensor
operator of a chiral boson CFT at that point Eling .

##  IV Discussion

In this paper we have shown that the incompressible Euler system in 2+1
dimensions can be described by a $U(1)$ gauge invariant action with an
explicit Chern-Simons term. Furthermore, we argued that this gauge theory
picture also can describe the so-called active scalar models which include the
surface quasi-geostropic equation. In the presence of a boundary, a gauge
theory includes edge states, which are physical degrees of freedom that live
on the boundary surface. We showed that the conserved Noether charge
associated with closed boundaries in our gauge theory exhibits a Kac-Moody
algebra, which indicates the presence of a chiral boson boundary theory with
an underlying conformal symmetry. Finally, we argued that zero magnetic field
(i.e., zero vorticity) nodal lines are natural boundaries in flows and that
fluctuations of these lines have a chiral boson description. This may explain
the mysterious appearance of conformal invariant measures of nodal lines and
the SLE description of these lines in the turbulent inverse cascade.

In future, it would be interesting to explore whether additional properties of
fluid systems can be encoded in gauge theory language and vice versa. For
example, it would be interesting to connect the gauge theory picture with
Arnold’s idea of the Euler equation as a geodesic flow on the infinite
dimensional Riemannian manifold of area preserving diffeomorphisms Arnold .
One observation is that in two dimensions linearized area preserving
diffeomorphisms in the space of the Lagrangian (co-moving) coordinates act as
U(1) gauge transformations Susskind:2001fb ; Sheikh-Jabbari:2023eba .

Another interesting question is whether can one extend the action to include
the fluid viscosity term, thereby giving a description of the full Navier-
Stokes equation? A related area of further work would be to investigate
whether the gauge theory description can shed more light on the mechanism and
properties of turbulence in 2d.

Finally, a natural area of investigation is whether there is a gauge theory
description of the 3+1 incompressible Euler equation. In this case vorticity
is a vector $\omega^{i}$ and the ideal vorticity equation has the form

$\displaystyle\partial_{t}\omega^{i}+v^{j}\partial_{j}\omega^{i}-\omega^{j}\partial_{j}v^{i}=0.$
(63)

For a 3+1 dimensional gauge theory the Bianchi identity is conservation of a
two-form

$\displaystyle\partial_{\mu}\left(\epsilon^{\mu\nu\rho\sigma}F_{\rho\sigma}\right)=0.$
(64)

These equations yield the familiar Maxwell equations

$\displaystyle\partial_{i}B^{i}=0$
$\displaystyle\partial_{t}B^{i}+\epsilon^{ijk}\partial_{j}E_{k}=0.$ (65)

Hence, we can identify the magnetic and electric fields as

$\displaystyle B^{i}$ $\displaystyle=\omega^{i}$ $\displaystyle
E^{i}=-\epsilon^{ijk}v_{j}\omega_{k}$ $\displaystyle=-\vec{v}\times\vec{B}.$
(66)

Consequently, the velocity is $\vec{v}=(\vec{E}\times\vec{B})/|B|^{2}$. The
next step, in principle, would be to find a Gauss law to relate the curl of
the velocity to the vorticity. However, a major problem in the construction of
the action principle is that there is no Chern-Simons term in 3+1 dimensions.
Therefore, a simple extension of our argument to one higher dimension
apparently will not work. Perhaps the mapping between fluids and gauge fields
is a part of the magic of 2+1 dimensions, but it may be possible to construct
a gauge theory description using different techniques.

## References

  * (1) P. Delplace, J. B. Marston, A. Venaille, “Topological Origin of Equatorial Waves”, Science 358 (2017) 1075.
  * (2) A. Souslov, K. Dasbiswas, M Fruchart, S. Vaikuntanathan, and V. Vitelli, “Topological waves in fluids with odd viscosity”, Phys. Rev. Lett. 122, 128001 (2019), [arXiv:1802.09649 [cond-mat.soft]]
  * (3) C. Tauber, P. Delplace, and A. Venaille, “A bulk-interface correspondence for equatorial waves”, [arXiv:1812.05488 [physics.flu-dyn]]
  * (4) M. Perrot, P. Delplace and A. Venaille, “Topological Transition in Stratified Fluids”, Nat.Phys. 15, 781 (2019 [arXiv:1810.03328 [physics.flu-dyn]]
  * (5) C. Tauber, P. Delplace, and A. Venaille, “Anomalous bulk-edge correspondence in continuous media”, Phys. Rev. Res. 2, 013147 (2020)[arXiv:1902.10050 [cond-mat.mes-hall]]
  * (6) G. M. Grad, H. Jud, and C. Tauber, “Topology in shallow-water waves: a violation of bulk-edge correspondence”, Comm. Math. Phys. 383, 731 (2021) [arXiv:2001.00439 [math-ph]].
  * (7) Y. Liu and Y. W. Sun, “Topological modes in relativistic hydrodynamics,” Phys. Rev. D 103 (2021) no.4, 044044 [arXiv:2004.13380 [hep-th]].
  * (8) Y. Liu and Y. W. Sun, “Topological hydrodynamic modes and holography,” Phys. Rev. D 105 (2022) no.8, 086017 [arXiv:2005.02850 [hep-th]].
  * (9) A. Venaille and P. Delplace, “Wave Topology Brought to the Coast”, Phys. Rev. Res 3, 043002 (2021), [arXiv:2011.03440v3 [physics.flu-dyn]]
  * (10) R. Green, J. Armas, J de Boer, and L. Giomi, “Topological waves in passive and active fluids on curved surfaces: a unified picture”, [arXiv:2011.12271 [cond-mat.soft]].
  * (11) C. Tauber and G. C. Thiang, “Topology in shallow-water waves: A spectral flow perspective”, Ann. Henri Poincare, (2022). [arXiv:2110.04097 [math-ph]].
  * (12) T. Matsuno, “Quasi-Geostrophic Motions in the Equatorial Area”, Journal of the Meteorological Society of Japan. Ser. II, Vol 44 (1966).
  * (13) M. Longuet-Higgins, “The Eigenfunctions of Laplace’s tidal equations over a sphere,” Phil. Trans. Royal Society of London 262: 1132: 511 (1968).
  * (14) S. C. Zhang, T. Hansson and S. Kivelson, “Effective-Field-Theory Model for the Fractional Quantum Hall Effect”, Phys. Rev. Lett. 62, 82 (1989).
  * (15) X. G. Wen and Q. Niu, “Ground-state degeneracy of the fractional quantum Hall states in the presence of a random potential and on high-genus Riemann surfaces”, Phys. Rev. B 41, 9377 (1990).
  * (16) D. Tong, “A Gauge Theory for Shallow Water,” [arXiv:2209.10574 [hep-th]].
  * (17) M. M. Sheikh-Jabbari, V. Taghiloo and M. H. Vahidinia, “Shallow Water Memory: Stokes and Darwin Drifts,”[arXiv:2302.04912 [hep-th]].
  * (18) H. Nastase and J. Sonnenschein, “Euler fluid in 2+1 dimensions as a gauge theory, and an action for the Euler fluid in any dimension,”[arXiv:2303.15229 [hep-th]].
  * (19) I. M. Held, R. T. Pierrehumbert, S. T. Garner, and K. L. Swanson, “Surface quasi-geostrophic dynamics,” J. Fluid Mech. 282 (1995): 1-20.
  * (20) D. Bernard, G. Boffetta, A. Celani, and G. Falkovich, “Conformal invariance in two-dimensional turbulence,” Nature Physics 2, 2 (2006): 124 [arxiv: nlin/0602017 [nlin.CD]].
  * (21) D. Bernard, G. Boffetta, A. Celani, and G. Falkovich, “Inverse turbulent cascades and conformally invariant curves,” Phys. Rev. Lett. 98, 2 (2007): 024501 [arxiv: nlin/0609069 [nlin.CD]].
  * (22) V. Larichev and J. McWilliams, “Weakly decaying turbulence in an equivalent barotropic fluid,” Phys. Fluids, A 3:5 (1991) 938.
  * (23) T. Tao, “Conserved quantities for the surface quasi-geostrophic equation,” March 6 2014, https://terrytao.wordpress.com/2014/03/06/conserved-quantities-for-the-surface-quasi-geostrophic-equation/.
  * (24) X. G. Wen, “Theory of the edge states in fractional quantum Hall effects,” Int. J. Mod. Phys. B 6, 1711-1762 (1992) doi:10.1142/S0217979292000840
  * (25) D. Tong, “Lectures on the Quantum Hall Effect,” [arXiv:1606.06687 [hep-th]].
  * (26) S. Elitzur, G. W. Moore, A. Schwimmer and N. Seiberg, “Remarks on the Canonical Quantization of the Chern-Simons-Witten Theory,” Nucl. Phys. B 326, 108-134 (1989) doi:10.1016/0550-3213(89)90436-7.
  * (27) A. P. Balachandran, G. Bimonte, K. S. Gupta and A. Stern, “Conformal edge currents in Chern-Simons theories,” Int. J. Mod. Phys. A 7, 4655-4670 (1992) doi:10.1142/S0217751X92002106 [arXiv:hep-th/9110072 [hep-th]].
  * (28) P. Goddard and D. I. Olive, “Kac-Moody and Virasoro Algebras in Relation to Quantum Physics,” Int. J. Mod. Phys. A 1, 303 (1986) doi:10.1142/S0217751X86000149.
  * (29) P. H. Ginsparg, “APPLIED CONFORMAL FIELD THEORY,” [arXiv:hep-th/9108028 [hep-th]].
  * (30) R. H. Kraichnan, “Inertial ranges in two-dimensional turbulence,” The Physics of Fluids 10, 7 (1967) 1417.
  * (31) O. Schramm, “Scaling limits of loop-erased random walks and uniform spanning trees,” Israel Journal of Mathematics 118, 1 (2000) 221 [arxiv: math/9904022 [math.PR]].
  * (32) M. Bauer and D. Bernard, “2D growth processes: SLE and Loewner chains,” Phys. Rept. 432, 115-221 (2006) doi:10.1016/j.physrep.2006.06.002 [arXiv:math-ph/0602049 [math-ph]].
  * (33) G. Falkovich and S. Musacchio, “Conformal invariance in inverse turbulent cascades” [arXiv:1012.3868 [cond-mat.stat-mech]].
  * (34) C. Eling, work in progress.
  * (35) V. I. Arnold and B. A. Khesin, “Topological methods in hydrodynamics,” Appl. Math. Sci,, vol. 125, Springer-Verlag, New York, 1998.
  * (36) L. Susskind, “The Quantum Hall fluid and noncommutative Chern-Simons theory,” [arXiv:hep-th/0101029 [hep-th]].


# PRTT: Precomputed Radiance Transfer Textures

Sirikonda Dhawal  CVIT, KCIS, IIIT-HyderabadHyderabadTelanganaIndia500032
<EMAIL_ADDRESS>,  Aakash KT  CVIT, KCIS, IIIT-
HyderabadHyderabadTelanganaIndia500032<EMAIL_ADDRESS>and  P.J.
Narayanan  CVIT, KCIS, IIIT-HyderabadHyderabadTelanganaIndia500032
<EMAIL_ADDRESS>
###### Abstract.

Precomputed Radiance Transfer (PRT) can achieve high quality renders of glossy
materials at real-time framerates. PRT involves precomputing a $k$-dimensional
transfer vector of Spherical Harmonic (SH) coefficients at specific points for
a scene. Most prior art precomputes transfer at vertices of the mesh and
interpolates color for interior points. They require finer mesh tessellations
for high quality renderings. In this paper, we explore and present the use of
textures for storing transfer. Using transfer textures decouples mesh
resolution from transfer storage and sampling which is useful especially for
glossy renders. We further demonstrate glossy inter-reflections by
precomputing additional textures. We thoroughly discuss practical aspects of
transfer textures and analyze their performance in real-time rendering
applications. We show equivalent or higher render quality and FPS and
demonstrate results on several challenging scenes.

Transfer, fragment-shader, Visibility, Textures

††copyright: none††ccs: Computing methodologies Rasterization††ccs: Computing
methodologies Ray tracing††ccs: Computing methodologies Visibility Figure 1.
We present transfer textures to decouple mesh resolution from transfer storage
and sampling. Our method precomputes and stores transfer in a texture that is
sampled in a fragment shader. The above scene has a minimally tessellated rug,
floor and table (shown in wireframe insets). Traditional triple product method
on vertex shader (left) is unable capture shadows due to insufficient sampling
of the transfer function. In contrast transfer textures run on the fragment
shader and accurately captures all details for the same tessellation. Note
that the scene has spatially varying color and roughness.

##  1\. Introduction

Ray Tracing is the method of choice for high-fidelity image generation.
However, it is computationally expensive for real-time applications where only
a few milliseconds of rendering time is available per-frame. Precomputed
Radiance Transfer (PRT) offloads expensive computations of ray tracing to a
pre-computation step, after which the stored data can be utilized for real-
time photorealistic rendering. PRT uses Spherical Harmonic (SH) lighting
[Ramamoorthi, 2009] to efficiently store and render complex effects in both
video games and offline rendering for movie production [Pantaleoni et al.,
2010]. Recently, PRT has been extended to efficiently handle area light
sources [Belcour et al., 2018; Wang and Ramamoorthi, 2018; Wu et al., 2020]
further increasing its utility.

The PRT framework proposed by [Sloan et al., 2002] pre-computes the transfer
function and stores it in SH basis at vertices of the scene, possibly with
compression [Sloan et al., 2003]. For a band $l$ SH projection with $k=l^{2}$
coefficients, this amounts to storing a $k$-dimensional vector for diffuse and
$k\times k$-dimensional matrix for glossy materials at each vertex. Using the
Triple Product formulation [Ng et al., 2004], it is possible to still store a
$k$-dimensional vector per-vertex for glossy materials by additionally storing
a global three-dimensional tripling coefficient matrix. A further modification
of triple products fixes the light [Sloan, 2008] and requires the storage of a
two-dimensional global matrix. The advantage of this method is that it retains
a compute complexity of $O(k^{2})$ as opposed to $O(k^{5/2})$ of traditional
triple products. Recently, the work of [Xin et al., 2021] reduces this time
complexity to $O(k^{3/2})$, although a real benefit is gained only for large
$k$.

The above PRT approaches store the transfer vectors/matrices on vertices of
the mesh. While rendering, color is evaluated at each vertex and interpolated
for internal points. This necessitates a reasonably dense mesh tessellation
(high mesh resolution) for high-quality renders. There exists few methods like
textured hierarchical PRT [McKenzie Chapter, 2010] and PRT of D3D9 framework
[Microsoft, 2003] which leverage the continuous texture space to store
transfer. The work of [Iwanicki and Sloan, 2009] also uses texture space
similar to [McKenzie Chapter, 2010] with a specific focus on shadows. These
methods focus only on diffuse reflection and do not demonstrate glossy
reflection and inter-reflections with textures. Directly extending above
mentioned methods for glossy renders requires $k^{2}$ coefficients per each
texel resulting in heavy texture storage, because they use the original PRT
formulation of [Sloan et al., 2002]. These extensions are thus infeasible in
real-time scenarios (Table 1). Can methods like Textured-PRT be extended to
handle glossy materials, inter-reflections, etc., at real-time rendering
rates?

In this paper, we present precomputed radiance transfer textures (transfer
textures) to efficiently store the transfer values using $k$ coefficients per
texel for both diffuse and glossy reflections. Our method uses the triple
product formulation of PRT [Ng et al., 2004]. Transfer textures store more
finely sampled transfer values and can evaluate color for each fragment in a
fragment shader. This improves render quality even for coarsely tessellated
meshes (Fig. 1). They can also support high-quality local effects like glossy
renders, inter-reflections, and normal maps. Higher texture resolution results
in greater precomputation effort but higher render quality at fixed render
times. This provides a way to balance computation effort and rendering quality
and to trade one off for the other. Texture space techniques like mip-mapping
and texture sets can be used for greater efficiency. We describe methods to
correctly and efficiently compute transfer textures and show real-time
framerates and superior render quality at low mesh tessellations for several
scenes. We further formulate and demonstrate inter-reflections using transfer
textures by completely fixing the light. We also show transfer texture
facilitate local effects like normal mapping. Finally, we compare and analyze
run-times and storage against vertex-based approaches with the triple product
method.

##  2\. Related Work

#### Precomputed Radiance Transfer (PRT)

PRT was first proposed by [Sloan et al., 2002], where they projected
environment lighting and light transport to SH basis for dynamic lighting for
diffuse and glossy materials. Since then, PRT and SH have received lot of
attention to efficiently compute SH basis [Snyder, 2006], efficient rotation
of SH [Nowrouzezahrai et al., 2012], compressing SH basis [Sloan et al.,
2003], microfacet BRDFs [Lehtinen and Kautz, 2003] and extending PRT for
dynamic scenes [Zhou et al., 2005]. More recently, [Wang and Ramamoorthi,
2018] extend PRT to support area lights achieving real-time frame rates for a
few light sources. This work was later extended to support a large number of
area lights while maintaining real-time frame rates [Wu et al., 2020]. All of
these methods were either orthogonal to the core PRT framework or extended the
core framework to support additional scenarios. In contrast, we improve the
core PRT framework to compute and store transfer on a texture instead of at
vertices.

#### Triple Products.

Triple products naturally arise in computer graphics in the rendering
equation. Triple products in wavelet basis and spherical harmonics have been
studied in depth [Ng et al., 2004]. Today, state-of-the art in PRT uses triple
products for dynamic relighting for diffuse and glossy scenes. By itself,
triple product method has a compute complexity of $O(k^{3})$. This method can
be made more computationally efficient by fixing the lighting [Sloan, 2008].
Specifically, triple products with fixed lighting in SH based PRT achieves a
compute complexity of $O(k^{2})$ and per-vertex storage of $k$-dimensional
vector. We augment the triple product method with transfer textures and
demonstrate superior rendering quality and real-time framerates.

#### Transfer stored on Textures.

Storage of transfer on a texture was first suggested by [Sloan et al., 2002].
[McKenzie Chapter, 2010; Iwanicki and Sloan, 2009] formally demonstrated
transfer storage on textures. These methods mainly focused on diffuse
materials without inter-reflections. In this work, we focus primarily on
glossy materials and also demonstrate inter-reflections with transfer
textures.

##  3\. Background on PRT with Triple Products

To make our document self-contained, we first review and discuss Precomputed
Radiance Transfer with triple products, which is the current state of the art
in PRT. The inspiration for the choice of using triple product formulation is
discussed in Sect. 6.3. The rendering equation for direct lighting at point
$p$ is given by:

(1)
$B^{p}(\omega_{o})=\int_{\Omega}L(\omega_{i})\rho^{p}(\omega_{o},\omega_{i})V^{p}(\omega_{i})(\omega_{i}\odot
n)d\omega_{i},$

where $\omega_{o}$ is the direction towards the viewer from $p$, $\omega_{i}$
is the incoming direction on the unit hemisphere $\Omega$ and $n$ is the
surface normal at $p$. $B^{p}$ is the reflected radiance in direction
$\omega_{o}$, $L$ is the incoming environment light from $\omega_{i}$, $V^{p}$
is the binary visibility function and $\rho^{p}$ is the Bi-directional
Reflectance Distribution Function (BRDF). Eq. 1 is decomposed into the
lighting $L$ and transfer $T^{p}(\omega_{i})=V^{p}(\omega_{i})(\omega_{i}\odot
n)$ which are then projected to the SH basis with coefficients $T^{p}_{i}$ and
$L_{i}$ respectively. Triple products [Ng et al., 2004] formulate transferred
radiance as:

(2)
$L^{p}_{k}=\int_{S^{2}}y_{k}(\omega)\left(\sum_{i=1}^{n^{2}}T^{p}_{i}y_{i}(\omega)\right)\left(\sum_{j=1}^{n^{2}}L_{j}y_{j}(\omega)\right)d\omega=\sum_{ij}\tau_{ijk}T^{p}_{i}L_{j},$

$B^{p}$ where $\tau_{ijk}$ is the triple product tensor (tripling coefficient
matrix). The final color is calculated by convolution of $L_{k}^{p}$ with BRDF
coefficients and evaluation at reflection direction. The above formulation for
PRT results in a $k$-vector for transfer stored at each point and $O(k^{5/2})$
compute [Ng et al., 2004] required for rendering. The compute efficiency can
be further improved to $O(k^{2})$ by fixing the light and precomputing a
global product matrix: $(M)_{ik}=\sum_{j}\tau_{ijk}L_{j}$ [Sloan, 2008].

##  4\. Transfer Textures

In this section, we begin with a description of computing and storing a band
$l$ SH projection of transfer on a texture (Sec 4.1). Next, we show how inter-
reflections can be pre-computed and incorporated in our framework (Sec 4.2).
Our implementation is described in Sec 5. Our approach achieves real-time
frame rates and better render quality, especially on low tessellation meshes,
as shown in Sec 6.

Input: $\mathcal{M},w,h,l$: Mesh $\mathcal{M}$, width $w$ & height $h$, SH
band $l$.

Output: $T_{o}$: Precomputed transfer texture

$T_{o}$ $\leftarrow$ Texture($w$, $h$, $l$)

// Init. texture.

$\mathcal{G}$ = OpenGL($\mathcal{M}$)

// G-Buffer

1

2for  _t in $T_{o}$_ do

3 point = $\mathcal{G}$[t.x][t.y].vertex

4 normal = $\mathcal{G}$[t.x][t.y].normal

$V$ = ComputeTransfer(point, normal)

// Path tracing

5 $V_{sh}$ = SHProject($V$)

6 $T_{o}$[t.x][t.y] = $V_{sh}$

7

$T_{o}$ = Dilate($T_{o}$, 3)

Algorithm 1 Pre-computing and storing the transfer texture.

###  4.1. Pre-computing Transfer Textures

The computation of transfer involves shooting multiple rays from a point $p$
in the scene and then evaluating and projecting the transfer to the SH basis.
For transfer textures, there are $N$ scene points $p$ corresponding to each
pixel $t$ in the texture. The mapping between $t$ and $p$ is defined by the UV
coordinates. To efficiently compute the transfer texture, we leverage
G-Buffers(Alg. 2) to interpolate vertex positions and normals based on their
corresponding UV-Coordinates (Alg. 1, line 2). Next, we read the G-buffer and
the scene geometry and evaluate the transfer function for each pixel in the
buffer (Alg. 1, lines 4-6). The transfer obtained is then projected to SH
basis and stored at the same pixel location in a initially empty texture
$T_{o}$ (Alg. 1, lines 7-8). Finally, $T_{0}$ is dilated to ensure that all
points inside a triangle receive a transfer value. At run-time, we fetch
transfer $T_{o}$ and use it with the triple product formulation to obtain
$B^{p}$.

1 Program  _VertexShader_ :

vec4 gl_Position;

// In-built variable

in vec3 p, n;

// Scene Point, Normal

in vec2 uv;

// UV co-ordinates

out vec3 vertex;

// Interpolated in Frag.

out vec3 normal;

// Interpolated in Frag.

2 void  _main()_ :

3 gl_Position = vec4(uv.x, uv.y, 0.0, 1.0);

4 vertex = p; normal = n;

5

6

7 Program  _FragmentShader_ :

8 in vec3 vertex;

9 in vec3 normal;

out vec4 gPos;

// G-Buffer

out vec4 gNorm;

// G-Buffer

10 void  _main()_ :

11 gPos = vec4(vertex,1.0);

gNorm = vec4(normal, 1.0); /* _w_ $\rightarrow$ alpha channel */

12

13

Algorithm 2 G-Buffer pass.(Vertex and fragment code)

###  4.2. Pre-computing transfer textures for inter-reflections

Inter-reflected radiance $B_{i}^{p}$ at point $p$ can be modeled as:

(3)
$B_{i}^{p}(\omega_{o})=\int_{\Omega}(1-V^{p}(\omega_{i}))B^{pq}(x,\omega_{i})\rho^{p}(\omega_{o},\omega_{i})(\omega_{i}\odot
n)d\omega_{i},$

where $B^{pq}$ is the radiance from a secondary hit-point $q$ towards $p$ and
$B_{i}^{p}$ is the inter-reflected radiance [Sloan et al., 2002]. First, we
factor out $1-V^{p}(x,\omega_{i})$ by only integrating over rays which hit
some geometry. For a scene point $p_{1}$ and a secondary hit $q_{1}$, the
radiance $B^{p_{1}q_{1}}$ can easily be precomputed given a zero-bounce
transfer texture $T_{o}$ from Alg. 1 (See Fig. 2).

Figure 2. Inter-reflections: The radiance $B^{p_{1}q_{n}}$ (red lines) towards
a point $p_{1}$ from a secondary hit-point $q_{n}$ can be computed by first
fetching transfer at $q_{n}$ using the zero-bounce transfer texture $T_{o}$,
applying the light $L$ followed by convolution with the BRDF at $q_{n}$ and
evaluation at reflected direction along the normal at $q_{n}$.
$B^{p_{1}q_{n}}$ forms an indirect environment map which is projected to SH
and stored at $p_{1}$ in an additional texture.

The radiance from $q_{1}$ towards $p_{1}$ is obtained using the triple product
formulation by fetching $T_{o}$ to obtain transfer at $q_{1}$.

This is done for all hit-points from $p_{1}$. This radiance now forms an
indirect environment map for the point $p_{1}$, which is then projected to SH
basis resulting in a $k$-vector $B^{pq}_{i}$, which is stored in a separate
one-bounce inter-reflection texture $T_{1}$. At run-time, the inter-reflected
radiance is obtained by convolving $B^{pq}_{i}$ fetched from $T_{1}$ with the
BRDF SH $\rho^{p}_{i}$ and evaluating at the reflection direction. The final
color is given as: $B^{p}(\omega_{o})+B_{i}^{p}(\omega_{o})$. Alg. 1 can be
easily extended to compute the second bounce texture $T_{2}$ and so on. The
number of textures required is linear in the number of bounces in this
setting, and the final color is just their summation.

###  4.3. Handling dense UV-packing

In the previous section we described methods for efficient computation of
transfer textures. Usage of these textures requires UV co-ordinates each
vertex to be defined. To obtain UV unwrapping of scene geometry, we used
_Smart UV-Unwrap_ or _Light Map Pack_ from Blender 3D [Blender, 2021]. One
caveat with UV unwrapping is that dense packing of UV islands may cause
overlaps which manifest as rendering artefacts. _Smart UV-Unwrap_ does not
guarantee non-overlapping islands while _Light Map Pack_ leads to texture
wastage and tiny pixel coverage for some parts of the geometry. In such
scenarios, texture-sets are beneficial.

Figure 3. Texture sets: TRM:Two Roza, one Monkey, We demonstrate the use of
texture-sets with TRM(right) where 4 small textures are assigned to each piece
of geometry against the use of single texture in case of TRM(left). The
artefacts can be seen clearly in the Monkey’s eyes and Roza’s hair as depicted
in insets. Note that in both cases, the memory requirements are the same
(single $1024\times 1024$ texture v/s four $512\times 512$ textures).

Consider an example scene as shown in Fig. 3. This scene contains 441K
triangles, all of which are packed into a single $1024\times 1024$ texture
(Fig. 3, left). As shown in the insets, this leads to artefacts. A better
approach is to use texture-sets, which means assigning individual textures to
each object in the scene (Fig. 3, right). In this case, each UV island can
occupy the entire space of the texture thus eliminating artefacts.

##  5\. Implementation Details

We implement Alg. 2 in Python using the ModernGL [Dombi, 2020] framework. We
generate and store the resulting G-buffers for each scene in a pre-process
step. Alg. 1 is implemented in Python and uses Embree [Wald et al., 2014] for
efficient ray intersection tests. We project to band $l=5$ (25 coefficients)
real spherical harmonics. As mentioned in Sect. 4.1 dilation is required to
ensure that all points in the scene receive a transfer value. Experimentally,
we found a dilation of three to be sufficient which may need adjustment
depending on the scene complexity. The time taken for generating transfer
textures for a scene like in Fig. 1 is approximately three hours.

Our real-time renderer is also implemented in the ModernGL framework. We
implement the triple product (TP) and triple product with fixed light (TPFL)
methods augmented with our transfer textures. Rendering is done in the
fragment shader using the generated transfer textures for the respective
scene. We render all scenes with glossy materials with spatially varying
roughness on a workstation with an NVIDIA RTX 3090 with a resolution of
1920$\times$1080\. An important detail is that we use the early depth pass to
prune fragments that are not visible thus avoiding unnecessary computations.
We use a texture resolution of 1024$\times$1024 texture as we have found it to
be best trade-off in between memory and quality for our scenes.

Table 1. Scene configurations. We list all scenes used in this paper, with their corresponding number of triangles and FPS with triple product (TP) and triple product fixed light (TPFL) methods on both vertex and fragment shaders (with transfer textures). Our approach achieves real-time framerates on all scenes. Scene | # tris. | Vert. (Trad.) | Frag. (Ours)
---|---|---|---
|  | TP | TPFL | TP | TPFL
Dragon (Fig 4) | 1.3M | 3.62 | 41.2 | 5.2 | 151.2
TRM (Fig 4) | 441K | 10.2 | 116.2 | 15.2 | 202.9
Room (Fig 1) | 21K | 352.3 | 2432.7 | 83.6 | 568.2
Plants (Fig 4) | 18K | 363.2 | 2597.6 | 6.7 | 168.3

##  6\. Results & Evaluation

Figure 4. We show results of TP and TPFL on the fragment shader using our
transfer textures (Bottom). We compare renderings with traditional vertex
shader based approaches on the top. For minimal tessellations, our method
accurately renders shadows whereas previous methods are unable due to
insufficient sampling of transfer. The third row (high-tessellation) shows
that renderings using traditional methods approximately approach the quality
of transfer textures on addition of more vertices. Note that low-FPS in case
of Dragon low-tessellation and TRM low-tessellation in vertex based TP and
TPFL is due to their high resolution geometry.

In this section, we present glossy rendering results including inter-
reflections using transfer textures on the fragment shader. We compare the
renderings with traditional vertex shader based approaches. We also discuss
and demonstrate the use of normal maps with transfer textures which is not
possible with traditional vertex based PRT. Finally, we analyze the memory
requirements and give a lower bound of FPS for tranfer texture usage in a
fragment shader. Rendering results are demonstrated on four scenes whose
statistics and performance comparisons are given in Table 1.

###  6.1. Glossy rendering & Inter-reflections

Fig. 4 shows the renders for three scenes: Plants, Dragon and TRM (two Roza,
one Monkey). All scenes have a ground plane, which is minimally tessellated,
as shown in the wireframe insets. The TP and TPFL methods on vertex shader are
unable to capture proper shadows on the ground plane due to sparse sampling of
the transfer function. In contrast, the TP method on the fragment shader using
our transfer textures properly reproduces shadows on the plane, albeit at a
very low FPS. The TPFL method with transfer textures also achieves a similar
render quality at a higher FPS. We note that the TP/TPFL methods on vertex
shader approach the render quality of our transfer textures with a highly
tessellated ground plane, as shown in the high-tessellation renderings. We
note that this requires the addition of _redundant_ vertices. We further note
that such situations frequently arise in production, for example with walls in
a room or any large surface with minimal curvature (Fig 1). In such cases, all
previous PRT methods on vertex shaders require the addition of avoidable
vertices to store the transfer on leading to drop in performance, as opposed
to our transfer textures method. Additional renders with different phong
exponents and environments maps for four different scenes are shown in Fig. 7
& 8.

Figure 5. We demonstrate inter-reflections with transfer textures on two
scenes: Diffuse Monkey (left) and glossy Roza (right). Renders with inter-
reflection maintain real-time frame-rates, albeit slightly lesser than zero-
bounce renderings.

Next, we demonstrate inter-reflections using transfer textures with the method
described in Sec 4.2. The zero-bounce and one-bounce renders with their
corresponding FPS are shown in Fig. 5 for two scenes: Monkey and Roza. Because
of extra texture fetch, convolution and evaluation operations the FPS with
inter-reflections is slightly lower, albeit still real-time. As described in
Sect. 4.2, additional bounces can be added with additional pre-computed
textures.

Figure 6. Normal Maps: Transfer textures make it possible to use the shading
normals from normal maps instead of the geometric normals. This is difficult
to achieve with traditinal vertex based PRT. The above scene shows a minimally
tessellated ground plane (wireframe, right) with a normal map. The scene is
rendered with TPFL and transfer textures. All normal map details are
preserved.

###  6.2. Normal Maps

Transfer textures make it possible to use normals maps during precomputation.
This translates to lesser vertices during rendering as finer detail can
instead be embedded in the normal map. Consider precomputation in traditional
vertex based PRT. In this case if a normal map is applied, it only ever
affects the transfer at those vertices thus loosing detail within each face
when using high frequency normal-texture. With transfer textures we can output
the shading normal from the normal map instead of the geometric normal in the
G-buffer during precomputation. In Alg. 1 line 6, the transfer will then be
computed at the shading normal instead. Since this texture is used to fetch
transfer during rendering, all normal map details are preserved. We show
renderings with normal maps in Fig. 6. The detail on the floor is due to the
normal map without any additional vertices, as can be seen in the wireframe
insets.

###  6.3. Memory Requirements

McKenzie demonstrated textures with diffuse PRT using Sloan’s formulation.
Consider directly implementing Sloan’s glossy formulation with textures
instead. This amounts to storing a $k\times k$ matrix per texel which quickly
becomes intractable, even for reasonably small textures. Thus augmenting the
triple product formulation to transfer textures is a clear choice. The memory
requirements for vertex as well as texture (fragment) based approaches is
shown in Table 2. The former’s memory requirements depend on the scene
complexity whereas it is constant for textures. Furthermore, a direct
extension of Sloan’s method to textures is infeasible, as shown in the fifth
column (2.5 GB per texture).

Table 2. Memory requirements for vertex based and transfer texture based approaches for a $1024\times 1024$ texture. Note that McKenzie uses Sloan’s approach which results in large textures for glossy rendering. Scene | # tris. | Vert. Mem. | Tex. Mem.
---|---|---|---
|  | Sloan | [Ng et al.] | McKenzie | Ours
Room | 21K | 64MB | 2.5MB | 2.5GB | 100MB
Dragon | 1.3M | 5.2GB | 215.8MB | 2.5GB | 100MB
TRM | 441K | 2.3GB | 139.3MB | 2.5GB | 100MB
Plants | 18K | 64MB | 2.5MB | 2.5GB | 100MB

###  6.4. Lower Bound on FPS

Since transfer textures are used in fragment shaders with an early depth pass,
we achieve a lower bound on the FPS. The computation is roughly the same for
each fragment and the worst case is when all fragments contain some geometry
to be processed and rendered. This is in contrast to vertex based approaches,
where run-time depends on the number of vertices in the scene. We demonstrate
this in Fig. 4 in the TRM (441K verts) and Dragon (1.3M verts) scenes. Here,
the FPS is lower for vertex based approach as compared to fragment based
approach with transfer texture, in both TP and TPFL.

Figure 7. Results of our transfer textures method on different light settings
Figure 8. Results of our transfer textures method with different Phong BRDFs
with same light

##  7\. Conclusions, Limitations & Future work

In this paper, we presented precomputed radiance transfer textures for
decoupling mesh tessellation from transfer sampling and storage for glossy
rendering. We described methods to efficiently and correctly compute these
textures and also demonstrated incorporation of inter-reflections using
additional precomputed textures. We compared our renderings with traditional
vertex based PRT approaches and thoroughly analyzed the memory requirements of
transfer textures. We demonstrated real-time framerates for rendering with
transfer textures on the fragment shader and superior render quality for
minimally tessellated meshes. Additionally, we gave a lower bound on the FPS
which will be useful in performance analysis in production. Our approach
inherits the advantages of texture based optimizations like textures-sets,
mip-maps and level of detail which can be easily incorporated. Although we
demonstrate on a fixed texture resolution, it can be tailored accordingly
depending on the hardware constraints and rendering quality needed. This is in
contrast to vertex based methods that provide vertex count as the only control
knob and little control over level of detail.

A limitation of transfer textures is that inter-reflections essentially bake
the lighting and BRDF i.e. they cannot be changed without re-computation. We
note that the work of [Sloan et al., 2002] also bakes BRDF (including albedo)
into their transfer matrices for inter-reflections. We would like to address
this issue for future extensions of this work.

## References

  * [1]
  * Belcour et al. [2018] Laurent Belcour, Guofu Xie, Christophe Hery, Mark Meyer, Wojciech Jarosz, and Derek Nowrouzezahrai. 2018.  Integrating Clipped Spherical Harmonics Expansions.  _ACM Trans. Graph._ 37, 2, Article 19 (mar 2018), 12 pages.  https://doi.org/10.1145/3015459
  * Blender [2021] Blender. 2021\.  _Blender - a 3D modelling and rendering package_.  Blender Foundation, Blender Institute, Amsterdam.  http://www.blender.org
  * Dombi [2020] Szabolcs Dombi. 2020\.  ModernGL, high performance python bindings for OpenGL 3.3+.  https://github.com/moderngl/moderngl.
  * Iwanicki and Sloan [2009] Michal Iwanicki and Peter-Pike Sloan. 2009.  Normal Mapping with Low-Frequency Precomputed Visibility. In _SIGGRAPH 2009: Talks_ _(SIGGRAPH ’09)_. ACM, New York, NY, USA, Article 52, 1 pages.  https://doi.org/10.1145/1597990.1598042
  * Lehtinen and Kautz [2003] Jaakko Lehtinen and Jan Kautz. 2003.  Matrix Radiance Transfer. In _Proceedings of the 2003 Symposium on Interactive 3D Graphics_ _(I3D ’03)_. Association for Computing Machinery, New York, NY, USA, 59–64.  https://doi.org/10.1145/641480.641495
  * McKenzie Chapter [2010] Harrison Lee McKenzie Chapter. 2010\.  Textured Hierarchical Precomputed Radiance Transfer.  (2010).
  * Microsoft [2003] Microsoft. 2003\.  _Microsoft DirectX 9 Programmable Graphics Pipeline_.  Microsoft Press, USA.
  * Ng et al. [2004] Ren Ng, Ravi Ramamoorthi, and Pat Hanrahan. 2004.  Triple Product Wavelet Integrals for All-Frequency Relighting.  _ACM Trans. Graph._ 23, 3 (Aug. 2004), 477–487.  https://doi.org/10.1145/1015706.1015749
  * Nowrouzezahrai et al. [2012] Derek Nowrouzezahrai, Patricio Simari, and Eugene Fiume. 2012\.  Sparse Zonal Harmonic Factorization for Efficient SH Rotation.  _ACM Trans. Graph._ 31, 3, Article 23 (June 2012), 9 pages.  https://doi.org/10.1145/2167076.2167081
  * Pantaleoni et al. [2010] Jacopo Pantaleoni, Luca Fascione, Martin Hill, and Timo Aila. 2010\.  PantaRay: Fast Ray-Traced Occlusion Caching of Massive Scenes.  _ACM Trans. Graph._ 29, 4, Article 37 (July 2010), 10 pages.  https://doi.org/10.1145/1778765.1778774
  * Ramamoorthi [2009] Ravi Ramamoorthi. 2009\.  _Precomputation-Based Rendering_.  NOW Publishers Inc.
  * Sloan [2008] Peter-Pike Sloan. 2008\.  Stupid spherical harmonics (sh) tricks. In _Game developers conference_ , Vol. 9. 42.
  * Sloan et al. [2002] Peter-Pike Sloan, Jan Kautz, and John Snyder. 2002\.  Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments.  _ACM Trans. Graph._ 21, 3 (July 2002), 527–536.
  * Sloan et al. [2003] Peter-Pike J. Sloan, J. Hall, J. Hart, and John M. Snyder. 2003\.  Clustered principal components for precomputed radiance transfer.  _ACM SIGGRAPH 2003 Papers_ (2003).
  * Snyder [2006] John Snyder. 2006\.  _Code Generation and Factoring for Fast Evaluation of Low-order Spherical Harmonic Products and Squares_.  Technical Report MSR-TR-2006-53. 9 pages.
  * Wald et al. [2014] Ingo Wald, Sven Woop, Carsten Benthin, Gregory S. Johnson, and Manfred Ernst. 2014.  Embree: A Kernel Framework for Efficient CPU Ray Tracing.  _ACM Trans. Graph._ 33, 4, Article 143 (July 2014), 8 pages.  https://doi.org/10.1145/2601097.2601199
  * Wang and Ramamoorthi [2018] Jingwen Wang and Ravi Ramamoorthi. 2018.  Analytic Spherical Harmonic Coefficients for Polygonal Area Lights.  _ACM Trans. Graph._ 37, 4, Article 54 (July 2018), 11 pages.  https://doi.org/10.1145/3197517.3201291
  * Wu et al. [2020] Lifan Wu, Guangyan Cai, Shuang Zhao, and Ravi Ramamoorthi. 2020\.  Analytic Spherical Harmonic Gradients for Real-Time Rendering with Many Polygonal Area Lights.  _ACM Trans. Graph._ 39, 4, Article 134 (July 2020), 14 pages.  https://doi.org/10.1145/3386569.3392373
  * Xin et al. [2021] Hanggao Xin, Zhiqian Zhou, Di An, Ling-Qi Yan, Kun Xu, Shi-Min Hu, and Shing-Tung Yau. 2021.  Fast and Accurate Spherical Harmonics Products.  _ACM Trans. Graph._ 40, 6, Article 280 (dec 2021), 14 pages.  https://doi.org/10.1145/3478513.3480563
  * Zhou et al. [2005] Kun Zhou, Yaohua Hu, Stephen Lin, Baining Guo, and Heung-Yeung Shum. 2005.  Precomputed Shadow Fields for Dynamic Scenes.  _ACM Trans. Graph._ 24, 3 (July 2005), 1196–1201.  https://doi.org/10.1145/1073204.1073332


# gundapusunil at SemEval-2020 Task 9: Syntactic Semantic LSTM Architecture
for SENTIment Analysis of Code-MIXed Data

Sunil Gundapu
Language Technologies Research Centre
KCIS, IIIT Hyderabad
Telangana, India
<EMAIL_ADDRESS>
&Radhika Mamidi
Language Technologies Research Centre
KCIS, IIIT Hyderabad
Telangana, India
<EMAIL_ADDRESS>

###### Abstract

The phenomenon of mixing the vocabulary and syntax of multiple languages
within the same utterance is called Code-Mixing. This is more evident in
multilingual societies. In this paper, we have developed a system for SemEval
2020: Task 9 on Sentiment Analysis for Code-Mixed Social Media Text. Our
system first generates two types of embeddings for the social media text. In
those, the first one is character level embeddings to encode the character
level information and to handle the out-of-vocabulary entries and the second
one is FastText word embeddings for capturing morphology and semantics. These
two embeddings were passed to the LSTM network and the system outperformed the
baseline model.

00footnotetext:  This work is licensed under a Creative Commons Attribution
4.0 International Licence. Licence details:
http://creativecommons.org/licenses/by/4.0/.

##  1 Introduction

Code-Mixing is a phenomenon which is evident in multilingual societies (Shana
Poplack et al., 2003). It reflects the use of distinct grammatical systems and
vocabulary of the languages being used simultaneously in a single utterance or
conversation. This technique used in communication commonly is widely found
today in popular social media platforms like Twitter, Facebook, Instagram in
the form of posts, comments, replies, especially in chats. This is evident in
multilingual societies like India, Canada, Ireland, South Africa, Switzerland,
and many others.

India has officially recognized 22 regional
languages111https://en.wikipedia.org/wiki/Languages_of_India. So, in
multilingual societies like India, most of the social media users
predominantly integrate the well-known language, like English, with their
native languages. 560 million Internet
users222https://www.internetworldstats.com/stats7.htm in India exchange
information by mixing their regional languages with prominent language like
English, which produces a huge amount of code-mixed social media corpus. One
such trending combination is the mixing of Hindi and English with the output
in Hinglish333https://en.wikipedia.org/wiki/Hinglish (Hi-En) code-mixed data.
Consider the example sentence which illustrates the code-mixing phenomenon
being addressed in this paper. “Congratulations/Eng Sir/Eng Ji/Hin Dobara/Hin
PM/Eng banee/Hin ki/Hin hardik/Hin subhkamnaye/Hin aapko/Hin ./O”.
(Translation into English: Congratulations sir, Best wishes to become Prime
Minister again). The words followed by language tags /Hin, /Eng, and /O
correspond to Hindi, English and Other respectively.

Sentiment analysis of code-mixed data has become a prominent research area in
recent times in the field of NLP. But identifying the sentiment in code-mixed
data is hard since it poses the following challenges: (i) Romanized code-mixed
data is noisy and ambiguous in nature. (ii) Accessible datasets are smaller in
size to tune neural networks. (iii) Frequent occurence of non-standard
spellings (such as pyaaarr, goooood). (iv) The phrase/word contractions (cmng
for coming, IDK for I don’t know). (v) Spelling variations. A single word
pyaar (love), can be written as “piyar”, “pyaarrrr”, “peyar”, “pyar”, or
“piyaar”, etc. To handle these challenges we postulate that FastText
embeddings enrich the word vectors with sub-word information and that
character level embeddings should be able to assist deep learning models to
handle unknown Hindi words.

In this paper, we propose models to predict the sentiment label of a given
code-mixed tweet/text. The sentiment labels are positive, negative, or neutral
and the code-mixed languages selected are Hindi and English. This task is
conducted in CodaLab444https://competitions.codalab.org/competitions/20654
website and our CodaLab username is gundapusunil. All our models are trained
using only the trained dataset provided by SentiMix organizers. We started
experiments with traditional machine learning models like Logistic Regression,
Support Vector Machines but the f1-score on development set was below 0.63
then we moved to complex models like Long Short Term Memory (LSTM) based
models with different types of word embeddings where we were able to conquer
the baseline model.

Our paper is divided into the following sections: We begin with an
introduction to code-mixing and its challenges in Section 1. Related work of
various code-mixing strategies demonstrated in Section 2. Section 3 present
the description of SentiMix dataset. We then discuss the pre-processing steps
and compare machine learning and deep learning approaches with baseline model
results in Section 4. The results are reported in Section 5, and Section 6
concludes the paper.

##  2 Related Work

Several studies have been made in the areas of sentiment analysis and code-
mixed data. One of the earlier studies on code-mixed data was proposed by Gold
(1967) with the goal of language identification in which it was stated that
the structure of the language is procured by learning the language structure
from the given text and informant. Braj B., Kachru (1976) described the
structure of multilingual languages and language dependency in linguistic
convergence of code-mixing from an Indian Perspective. SentiWordNet for
English language introduced by Esuli and Sebastiani (2006), became the primary
source for all sense based lexical analysis and opinion classification.

Later, researchers have extended the work on machine learning and sentiment
analysis methods for Indian languages. In their study Siersdorfer, Chelaru et
al. (2010) used the SVM and Naïve Bayes classifiers to label millions of
comments for sentiment polarity. R. Sharma and P. Bhattacharyya (2014)
developed a lexicon-based sentiment analyzer for product reviews and the same
subjective lexicon-based model has been extended to Punjabi language. For
Malayalam movie reviews D. S. Nair, J. P. Jayan et al. (2014, 2015) initially
came up with a rule based system for sentiment analysis, later improved the
system with the support of the machine learning algorithms. MIKE 2015
sentiment analysis task for Hindi and Bengali used the Multinomial Naïve Bayes
classifier with the features for building the vector space constrained by
filtering the words based on WordNet.

In recent years, researchers have seen a huge improvement in the task of
sentiment analysis of English as well as Hinglish using deep neural networks.
M.G. Jhanwar, A. Das (2018) proposed an ensemble of character-trigrams based
LSTM model and a n-grams based Multinomial Naïve Bayes model to classify the
sentiments of Hinglish code-mixed data. Shalini K, Barathi Ganesh HB et al.
(2018) addressed the performance of distributed representation methods in
sentiment analysis and reported comparisons among different machine learning
and deep learning techniques. Other attempts include using sub-word level
compositions with LSTMs to capture sentiment at the morpheme level (Joshi et
al., 2016). We attempted to perform SemEval 2020 task-9 (Patwa et al., 2020)
with various classification and deep learning models to analyze the results
and also how such models contributed to a great advance in this task.

##  3 Dataset

In this paper, we used the dataset provided by Task 3: SentiMix in SemEval
2020. The corpus contains a total of 20000 tweets and it is sub-divided into
three sets (train, validation and test). Each corpus except test contains
code-mixed tweets along with their corresponding sentiment labels. These code-
mixed tweets are tokenized into tokens. And the tokens of each tweet are
separated by a new line. Each token is manually annotated with a language
identification tag which are: Hindi (Hin), English (Eng), Other (O).

Dataset | Positive | Negative | Neutral | Total
---|---|---|---|---
Train | 4634 | 4102 | 5264 | 14000
Valid | 982 | 890 | 1128 | 3000
Test | 1000 | 900 | 1100 | 3000
|  |  |  | 20000
Table 1:  Dataset Statistics.

Table 1 shows the distribution of sentiment classes in the SentiMix dataset.
Table 2 shows some examples of code-mixed tweets from the SentiMix dataset.
Here, the first column contains Hinglish tweet, the second column contains
English translation of tweet, the third column contains sentiment label.

Code-Mixed Tweet Approximate English Translation Sentiment Label All the best
Team India Jeet ke aana Team India all the best, come back with win. Positive
Aap bhi aisa drama kr do kam se km You also do this drama and at least.
Negative Aisa PM naa hua hai aur naa hee hoga Neither there has been a PM like
him, nor there will be Positive

Table 2:  Example Code-Mixed Tweets.

##  4 System Architecture

In this section, we present our models that are trained and validated on the
SentiMix dataset described in the previous section. We compare our approach
with Machine Learning and Neural Network based baselines. The full code of
system architecture can be found at
GitHub555https://github.com/SunilGundapu/SENTIment-Analysis-on-Hindi-English-
Code-MIXed-Data.

###  4.1 Preprocessing of Code-Mixed tweets

Our code-mixed data consists of excessive noise in the form of punctuations,
Uniform Resource Locator (URL’s), few Devanagari script Hindi words, twitter
mentions, hashtags, stop words, etc. In the preprocessing step, we take a stab
to overcome the noise in the the data by remove/normalize the unnecessary
tokens. Figure 1 explains the code-mixed dataset pre-processing pipeline. The
input for the pipeline is a tokenized tweet and the output is a cleaned tweet.

Figure 1: Code-Mixed text preprocessing pipeline.

###  4.2 Classical Supervised Machine Learning Algorithms

To design the finest system for sentiment analysis in code-mixed data, we
begin our experiments with traditional machine learning algorithms like
Support Vector Machines (SVM), One-vs-rest classifier with Logistic Regression
(OvRLR), Random Forest Classifier (RFC) and Multilayer Perceptron (MLP). The
input for these methods is a single d dimensional feature vector of a single
code-mixed tweet.

We analyze the results of the above classical algorithms with the combination
of two types of vectors. (i) Word level term frequency-inverse document
frequency (tf -idf) vector and (ii) Glove word embeddings. For all tokens in
the code-mixed tweet a feature vector is created by averaging over d
dimensional Glove embeddings and also it is experimented with tf-idf weighted
averaging. The code-mixed tweet vector construction scheme is described below:

$\displaystyle featureVector_{tweet}$
$\displaystyle=\dfrac{\sum_{n=1}^{N}\textbf{tf-
idf}(token_{i})\times\textbf{Glove}(token_{i})}{total\,no.\,of\,tokens\,in\,tweet\,(N)}$

Empirically, we found that standard averaging of Glove and tf-idf gave better
results than normal tf-idf weighted averaging.

###  4.3 Deep Neural Networks

In this subsection, we describe the character and word embedding based deep
neural network called Syntactic and Semantic LSTM (SS-LSTM) that gave better
predictions on our corpus. Initially, We tried with Word2Vec (T. Mikolov et
al., 2013), GloVe (J. Pennington et al., 2014), FastText (Bojanowski et al.,
2016), Character embeddings for each word in the input code-mixed tweet. We
train a simple LSTM model using each of these embeddings to test the
effectiveness of these embeddings for sentiment classification. FastText and
Character level embeddings gave slightly better results than other embeddings.
By considering these results we modeled the SS-LSTM architecture given below.

####  4.3.1 Character Level Embeddings

Character level embeddings use a one-dimensional convolutional neural network
(1D-CNN) to find the numeric representation of words by looking at their
character-level compositions. 1D-CNN (Yoon Kim, 2014) is an algorithm capable
of handling unseen words and also extracting syntactic information from the
segments of input. The character embedding step converts tweet tokens into a
$\textit{{d}}\times\textit{{T}}$ matrix. d is the dimension of vector and T is
the number of tokens in code-mixed tweet.

####  4.3.2 Word Level Embeddings

For word level embeddings we used the Facebook’s FastText. The main advantage
of FastText embeddings is to capture the hidden knowledge about a language,
like word analogies or semantic. And it is looking into the internal structure
of words, which could be very useful for morphologically rich languages like
Hindi. The FastText enrich word vectors with subword information.

####  4.3.3 Model Architecture

Figure 2: System (SS-LSTM) Architecture.

We model the task of SentiMix as a multi-class classification problem where
given a code-mixed tweet, the model outputs probabilities of it belonging to
three output classes - Positive, Negative, and Neutral. The proposed system
architecture (SS-LSTM) is shown in Figure 2. The input tweet is fed into
1D-CNN and FastText. These two word embedding models generate two
$\textit{{d}}\times\textit{{T}}$ matrices, one is for the 1D-CNN and the other
for the FastText. Here the dimension of each matrix is
$\textit{{256}}\times\textit{{T}}$. These two word embedding matrices are
passed to two LSTM layers. One LSTM layer uses a character level embeddings,
whereas the other layer uses a FastText word embeddings. These two layers
learn syntactic and semantic feature representation and encode sequential
patterns in the tweet. And each LSTM layer gives a
$\textit{{128}}\times\textit{{1}}$ dimensions vector.

The character embedding LSTM layer output is $C\in{R^{128\times 1}}$ and the
FastText word embedding LSTM layer output is $W\in{R^{128\times 1}}$. These
two output feature representations are row-wise concatenated and the output
vector dimension is $O\in{R^{256\times 1}}$. The output vector O passed to a
fully connected network with one hidden layer which models interactions
between these features and outputs probabilities per sentiment class. We used
Keras neural network library to implement this model. $O\in{R^{256\times 1}}$

##  5 Results

A summary of results from various techniques on the SentiMix test dataset is
present in Table 3. SS-LSTM gave the best performance on f1-score for each
sentiment class as well as on average f1-score. Our results thus indicate that
combining syntactic and semantic representations in SS-LSTM outperforms
individual LSTM-Character and LSTM-FastText embedding models.

Model | Representations | Precision | Recall | f1-Score
---|---|---|---|---
SVM | TF-IDF avg | 0.6315 | 0.6373 | 0.6308
RFC | TF-IDF avg | 0.6260 | 0.6323 | 0.6261
OvRLR | TF-IDF avg | 0.6391 | 0.6426 | 0.6377
MLP | TF-IDF avg | 0.6505 | 0.6433 | 0.6454
SVM | TF-IDF and Glove avg | 0.6409 | 0.6384 | 0.6412
RFC | TF-IDF and Glove avg | 0.6311 | 0.6358 | 0.6356
OvRLR | TF-IDF and Glove avg | 0.6357 | 0.6393 | 0.6343
MLP | TF-IDF and Glove avg | 0.6485 | 0.6523 | 0.6445
LSTM | Word2Vec | 0.6565 | 0.6538 | 0.6523
LSTM | FastText | 0.6605 | 0.6533 | 0.6554
LSTM | Glove | 0.6585 | 0.6523 | 0.6545
LSTM | 1D-CNN | 0.6825 | 0.6610 | 0.6651
LSTM | BERT | 0.6800 | 0.6712 | 0.6742
SS-LSTM | FastText and 1D-CNN | 0.6819 | 0.6773 | 0.6758
Table 3:  Results on test data for Hindi-English.

Machine learning models with tf-idf feature representations gave the
approximate baseline results. We observe that the tf-idf weighted average of
GloVe performed better than the simple average of vectors. And we used the
grid search (Srivastava et al., 2014) to find the better hyper-parameters like
number of LSTM layers, learning rate, and the number of epochs. We used GPU
for training deep learning models.

##  6 Conclusion

In this paper, we experimented the code-mixed dataset with various machine
learning and deep learning models. We see that the LSTM models performed far
better than traditional ML methods. In the first phase of the SentiMix
competition (development set), we were able to achieve a score of 0.6357. But
in the second phase (test dataset), our best score was only 0.6758. After
competition we attain the f1-score of 0.6789 by changing few parameters like
learning rate and number of LSTM’s. Till now we handled problems like unseen
words, spelling variations, dataset imbalance, emojis, short form of words,
etc. In future work, we plan to focus on issues like free ordering of words in
sentence constructions, short sentences with unclear semantic structure, etc.
And we would like to explore more deep neural network architectures that can
capture sentiments in code-mixed data.

## References

  * []  Parth Patwa, Gustavo Aguilar, Sudipta Kar, Suraj Pandey, Srinivas PYKL, Björn Gambäck, Tanmoy Chakraborty, Thamar Solorio, and Amitava Das.  2020\.  Semeval-2020 task 9: Overview of sentiment analysis of code-mixed tweets.  In _Proceedings of the 14th International Workshop on Semantic Evaluation (SemEval-2020)_ , Barcelona, Spain, December.  Association for Computational Linguistics.
  * [Borschinger and Johnson2011]  Poplack, Shana and Walker, James.  2003\.  Pieter Muysken, Bilingual speech: a typology of code-mixing.  Cambridge: Cambridge University Press, Pp. xvi+306.  Journal of Linguistics. 39. 678 - 683. 10.1017/S0022226703272297.
  * [Gold and TR1967]  E. M. Gold and T. R. Corporation.  1967\.  Language identification in the limit, Inf. Control, vol 10, no 5, pp 447–474.
  * [Gold and TR2006]  Braj B. Kachru.  1978\.  Toward Structuring Code-Mixing: An Indian Perspective.  International Journal of the Sociology of Language, 16:27-46.
  * [Gold and TR2006]  A. Esuli, F. Sebastiani, and V. G. Moruzzi.  2006\.  SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining, Proc. Lr, vol 0, pp 417–422.
  * [Borschinger and Johnson2011]  S. Siersdorfer, S. Chelaru, W. Nejdl, and J. San Pedro.  2011\.  How useful are your comments?.  In _Proceedings of 19th International Conference on World Wide Web 2011_ , vol 15, pp 891–900.
  * [Borschinger and Johnson2011]  R. Sharma and P. Bhattacharyya.  2014\.  A sentiment analyzer for hindi using hindi senti lexicon.  In _Proceedings of 11th International Conference on Natural Language Processing 2014_ , p 150.
  * [Borschinger and Johnson2011]  K. Shalini, H. B. Ganesh, M. A. Kumar and K. P. Soman.  2018\.  Sentiment Analysis for Code-Mixed Indian Social Media Text With Distributed Representation.  _International Conference on Advances in Computing, Communications and Informatics (ICACCI)_ , Bangalore, pp 1126-1131.
  * []  D. S. Nair, J. P. Jayan, E. Sherly.  2015\.  Sentima-sentiment extraction for malayalam.  In _Advances in Computing, Communications and Informatics (ICACCI), 2015 International Conference on. IEEE_ , pp 2381-2384.
  * []  K. Sarkar and S. Chakraborty.  2015\.  A sentiment analysis system for indian language tweets.  In _International Conference on Mining Intelligence and Knowledge Exploration._ , Springer, pp 694-702.
  * []  Madan G. Jhanwar and Arpita Das.  2018\.  _An Ensemble Model for Sentiment Analysis of Hindi-English Code-Mixed Data_ , CoRR, vol abs/1806.04450.
  * []  A. Joshi, A. Prabhu, M. Shrivastava, V. Varma.  2016\.  Towards sub-word level compositions for sentiment analysis of hindi english code mixed text.  In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_ , pp 2482-2491.
  * []  Patra, Braja and Das, Dipankar and Das, Amitava.  2018\.  Sentiment Analysis of Code-Mixed Indian Languages: An Overview of SAIL Code-Mixed Shared Task @ICON-2017.
  * []  Gupta, Umang and Chatterjee, Ankush and Srikanth, Radhakrishnan and Agrawal, Puneet.  2017\.  A Sentiment-and-Semantics-Based Approach for Emotion Detection in Textual Conversations.
  * []  A. Joulin, E. Grave, P. Bojanowski, and T. Mikolov.  2016\.  Bag of tricks for efficient text classification.  In _arXiv preprint arXiv:1607.01759_.
  * []  T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean.  2013\.  Distributed representations of words and phrases and their compositionality.  In _Advances in neural information processing systems_ , pp 3111-3119.
  * []  J. Pennington, R. Socher, and C. D. Manning.  2014\.  Glove: Global vectors for word representation.  In _EMNLP_ , vol 14, pp 1532–1543.
  * []  A. Joshi, A. Prabhu, M. Shrivastava, V. Varma.  2014\.  Convolutional Neural Networks for Sentence Classification.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pp 1746-1751.
  * []  Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.  2014\.  _Dropout: a simple way to prevent neural networks from overfitting_ , 15(1):1929–1958.