# Gravitational anomalies and entropy

Bibhas Ranjan Majhi
IUCAA, Post Bag 4, Ganeshkhind,
Pune University Campus, Pune 411 007, India
E-mail<EMAIL_ADDRESS>
###### Abstract

A derivation of entropy from the expressions for two dimensional gravitation
anomalies is given. Starting from the near horizon anomalous energy-momentum
tensors corresponding to particular anomalies, the Virasoro algebra with
central extension is obtained. The central charge is identified by comparing
with the standard form of the algebra. Then the conserved charge in the ground
state is computed. Finally, using the Cardy formula the entropy is obtained.
Here both the vector and chiral theories are discussed.

##  1 Introduction

Since the discovery of Hawking effect [1], there are several methods in the
literature to understand this. The important feature is that all of them yield
the universal result: temperature and entropy of a black hole are proportional
to the surface gravity and the horizon area, respectively [1, 2]. Still it
remains as a long standing and important question whether the black hole
thermodynamics has a statistical description in terms of microscopic states.
Till now there exists several ways towards this attempt: loop quantum gravity
[3], string theory [4], the conformal field theory [5, 6] technique, etc. But
none of them is complete.

After the discovery of the Hawking effect, people started thinking that it may
play a major role to shed some light towards the quantum nature of gravity.
Therefore it is necessary to have deeper understanding of this effect. Among
the existing approaches, anomaly method attracted a lot. The idea is that: it
is not possible to simultaneously preserve conformal and diffeomorphism
symmetries at the quantum level. In vector theory, any one of them must be
sacrificed. As the latter symmetry is usually retained, there is, in general,
a violation of the conformal invariance which is manifested by a non-vanishing
trace of the energy-momentum tensor and leads to trace anomaly [7]. Whereas in
the chiral theory both the trace and diffeomorphism anomalies exist [8, 9,
10]. Long ago, Christansen and Fulling [11] first determined the Hawking flux
in the vector theory for a two dimensional black hole.

Recently, the method has been extended to the higher dimensional black holes
($D\geq 4$). This is mainly based on the fact that near the event horizon the
effective theory is two dimensional and the background is effectively the two
dimensional ($t-r$) black hole metric [12, 13, 14]. Now since, the ingoing
modes are absorbed classically by the black hole, the theory must be chiral
near the horizon and at the quantum level both the trace and diffeomorphism
anomalies will appear. In this case, there exists two types of anomaly
expressions: consistent and covariant expressions. They are related by a local
counter term [8, 9, 10]. Robinson and Wilczek [12] showed that the Hawking
flux can be obtained from the consistent anomaly expression with the use of
the covariant boundary condition. In this case, both the Ward identities were
required. Later on, an elegant and conceptually clearer derivation was given
in [15] which was based on the covariant anomaly expression 111For more
details and several important issues, see [16, 17, 18, 19].. Interestingly,
only one Ward identity was required.

Although, the anomaly approach has been applied for several metrics to find
the emission flux, nevertheless, a connection between these anomalies and the
entropy has so far been missing in the literature. Such a investigation is
necessary to have complete understanding of the role played by the anomalies
in gravitational physics. In this paper, we establish this connection
precisely in the case of an arbitrary dimensional static black hole.

We briefly give our methodology to compute the entropy using the near horizon
gravitational anomalies. The method, we shall follow here, is the conformal
field theory (CFT) technique, particularly the work of Solodukhin [21]. It is
well known that the relevant conformal theory is the Liouville theory [20]
living on the two dimensional boundary. In this theory, the conserved charges
corresponding to near horizon diffeomorphism symmetry generators obey the
standard Virasoro algebra with central extension and it leads to central
charge [21] 222For a Hamiltonian or Lagrangian approach and its related issues
to find the Virasoro algebra, see [23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
34, 35, 36, 37, 38, 39, 40, 41].. Then evaluating the ground state conserved
charge one can find the entropy using Cardy formula [22]. Here, we shall first
cast the anomalous energy-momentum tensors corresponding to respective
theories (vector and chiral theories) to the identical form of that of
Liouville theory, since this is a relevant theory for the conformal field
theory [20]. This will be done by using the transformations of metric and the
auxiliary scalar field. Using this transformed tensor, the conserved charges
$L_{n}$, corresponding to the near horizon diffeomorphism symmetry generators
[23, 24], will be defined. Computation of the commutator among $L_{n}$ will
lead to the standard Virasoro algebra with the central extension. Then one can
immediately identify the central charge. The other important quantity is the
conserved charge corresponding to the zero mode, i.e. $L_{0}$. This will be
obtained by straight forward evaluation. We shall then apply the Cardy formula
[22] to obtain the expression for entropy. Generally two copies of Virasoro
algebra, one for $L_{n}$ (holomorphic) and other for ${\bar{L}}_{n}$ (anti-
holomorphic), contribute to the entropy. However, in our case since the theory
is near horizon where only ingoing modes contribute, and so only one copy,
namely $L_{n}$, is sufficient to calculate the entropy.

The organization of the paper is as follows. In the next section, the near
horizon effective two dimensional metric will be introduced. Here it will be
written in the null coordinates which will be relevant for our subsequent
analysis. In section 3, the energy-momentum tensor in two dimensional vector
theory will be casted to that of Liouville theory. Defining the conserved
charge, we will obtain the Virasoro algebra. Then by the Cardy formula the
entropy will be obtained. Next section will be devoted for the same in the
case of chiral theory. Final section will contain the conclusions.

##  2 Metric and null coordinates

The physics of a black hole is mainly dominated by the event horizon
properties. Near this horizon, the theory is effectively determined by the two
dimensional ($t-r$) metric [12, 13, 14]:

$\displaystyle ds^{2}=F(r)dt^{2}-\frac{dr^{2}}{F(r)}$ (1)

whose event horizon is defined by the relation $F(r=r_{H})=0$ and the surface
gravity is given by $\kappa=\frac{F^{\prime}(r_{H})}{2}$. For our later
analysis, it will be appropriate to use the null tortoise coordinates, defined
as,

$\displaystyle u=t-r^{*},\,\,\ v=t+r^{*};\,\,\,\ dr^{*}=\frac{dr}{F(r)}~{}.$
(2)

In these coordinates metric (1) takes to the following form

$\displaystyle ds^{2}=\frac{F(r)}{2}(dudv+dvdu)~{}.$ (3)

The justification of choosing the effective $2D$ metric (1) for our purpose of
finding entropy by two dimensional gravitational anomalies in the context of
Virasoro algebra is as follows. Consider first the following four dimensional
metric:

$\displaystyle
ds^{2}_{4}=F(r)dt^{2}-\frac{dr^{2}}{F(r)}-r^{2}(d\theta^{2}+\sin^{2}\theta
d\phi^{2})~{}.$ (4)

The horizon (apparent) is defined as the curve $H$ such that [42],

$\displaystyle g^{ab}\nabla_{a}r\nabla_{b}r|_{H}=0~{},$ (5)

where $g_{ab}$ is the ($t-r$) sector of the full metric (4). This clearly says
$F(r=r_{H})=0$, as stated earlier. Now the above condition is invariant under
conformal transformation $g_{ab}\rightarrow\Omega^{2}g_{ab}$ and so the two
dimensional metric is determined by this condition only upto a conformal
factor. Interestingly, very near to the horizon, the diffeomorphisms which
preserve the condition (5), form the infinite dimensional group of conformal
transformations in two dimensions. The corresponding generators satisfy
Virasoro algebra. Also, the fields become massless. For details, see section 2
of [21]. Therefore, the physics, near the horizon, is conformal and governed
by an effective two dimensional field theory. Thus we see that the horizon
defining condition (5) is essentially a condition on the radial function $r$
of the full metric (4) and the two dimensional metric is determined upto a
conformal factor. So the horizon dynamics is encoded in the radial function
while the $2$D metric is chosen from the same conformal class. Therefore, here
we have chosen the two dimensional metric as (1) and it is feasible to study
conformal field theory on this background.

##  3 Vector theory, Virasoro algebra and entropy

In this section, the entropy will be evaluated from the anomalous stress
tensor for the vector theory. The idea is that near the event horizon, the
theory is effectively dominated by the two dimensional metric of the form (3).
For details, see [12, 13, 14]. Now the energy-momentum tensor for the scalar
field under this effective two dimensional metric, at the quantum level, leads
to non-vanishing trace since usually the diffeomorphism symmetry is retained.
Such theory is called as vector theory.

In this case, the trace anomaly is given by [43]

$\displaystyle\tilde{T}{{}^{a}_{a}}=\frac{\tilde{R}}{24\pi}~{},$ (6)

and the energy-momentum tensor satisfies the conservation equation
$\tilde{\nabla}^{a}\tilde{T}_{ab}=0$. The anomaly induced effective action
$S_{P}$, obtained from the functional integration of
$\tilde{T^{a}_{a}}=\frac{2}{\sqrt{-\tilde{g}}}\tilde{g}^{ab}\frac{\delta
S_{P}}{\delta\tilde{g}^{ab}}$, is given by the Polyakov action [44],

$\displaystyle S_{P}=-\frac{1}{96\pi}\int
d^{2}x\sqrt{-{\tilde{g}}}{\tilde{R}}\frac{1}{\tilde{\Box}}{\tilde{R}}~{}.$ (7)

This action is non-local and it can be written in local form by introducing an
auxiliary field $\tilde{\Phi}$:

$\displaystyle S_{P}=-\frac{1}{96\pi}\int
d^{2}x{\sqrt{-\tilde{g}}}(-\tilde{\Phi}\tilde{\Box}\tilde{\Phi}+2\tilde{\Phi}\tilde{R})~{}.$
(8)

The two dimensional anomalous energy-momentum tensor is given by,

$\displaystyle{\tilde{T}}_{ab}=\frac{2}{\sqrt{-\tilde{g}}}\frac{\delta
S_{P}}{\delta\tilde{g}^{ab}}=\frac{1}{48\pi}\Big{[}\tilde{\nabla}_{a}\tilde{\Phi}\tilde{\nabla}_{b}\tilde{\Phi}-2\tilde{\nabla}_{a}\tilde{\nabla}_{b}\tilde{\Phi}+\tilde{g}_{ab}\Big{(}2\tilde{R}-\frac{1}{2}\tilde{\nabla}_{c}\tilde{\Phi}\tilde{\nabla}^{c}\tilde{\Phi}\Big{)}\Big{]}$
(9)

with the auxiliary scalar $\tilde{\Phi}$ satisfies the equation of motion,

$\displaystyle\tilde{\Box}\tilde{\Phi}=\tilde{R}~{}.$ (10)

Here $\tilde{R}$ is the two dimensional Ricci scalar. Christensen and Fulling
[11] evaluated the Hawking flux by solving the conservation equation and the
trace anomaly expression (6) for the Schwarzschild black hole. Here, in the
following, we will find the entropy from the expression for anomalous energy-
momentum tensor (9).

Now it is known that the Liouville theory is conformally invariant and
possesses two sets of Virasoro generators $L_{n}$ and $\bar{L}_{n}$ [20].
Therefore, it is a relevant CFT and so it is convenient to recast the energy-
momentum tensor (9) into that of the Liouville theory so that it fits into the
Virasoro scheme. The same technique was also adopted in [21], where the four
dimensional Einstein-Hilbert action was casted to Liouville action by
dimensional reduction technique and then imposing a set of particular
transformations. Motivated by this fact, here also we will do the same. This
is done by the following set of transformations,

$\displaystyle\tilde{g}_{ab}=e^{\Big{(}\frac{2}{qr_{H}}-\frac{qr_{H}}{16}\Big{)}\Phi}g_{ab};\,\,\,\
\tilde{\Phi}=\frac{qr_{H}}{8}\Phi~{}.$ (11)

Under these, (9) and (10) transform to

$\displaystyle
T_{ab}=\frac{1}{48\pi}\Big{[}\frac{1}{2}\nabla_{a}\Phi\nabla_{b}\Phi-\frac{qr_{H}}{4}\nabla_{a}\nabla_{b}\Phi+g_{ab}\Big{(}2R-\frac{1}{4}\nabla_{c}\Phi\nabla^{c}\Phi-\frac{4}{qr_{H}}\nabla_{c}\nabla^{c}\Phi$
$\displaystyle+\frac{qr_{H}}{8}\nabla_{c}\nabla^{c}\Phi\Big{)}\Big{]}$ (12)
$\displaystyle\Box\Phi=\Big{(}\frac{qr_{H}}{16}+\frac{2}{qr_{H}}\Big{)}^{-1}R$
(13)

where $r_{H}$ is the radius of the event horizon and $q$ is a constant.
Substituting (13) in (12), the energy-momentum tensor reduces to the following
form

$\displaystyle
T_{ab}=\frac{1}{48\pi}\Big{[}\frac{1}{2}\nabla_{a}\Phi\nabla_{b}\Phi-\frac{qr_{H}}{4}\nabla_{a}\nabla_{b}\Phi+g_{ab}\Big{(}\frac{qr_{H}}{4}\nabla_{c}\nabla^{c}\Phi-\frac{1}{4}\nabla_{c}\Phi\nabla^{c}\Phi\Big{)}\Big{]}~{}.$
(14)

This is similar to the energy-momentum tensor of Liouville theory as obtained
by Solodukhin [21] from the four dimensional Einstein-Hilbert action. Here, we
obtained this from the two dimensional trace anomaly expression by applying
the transformations (11).

Before proceeding further, note that because of non-vanishing of trace

$\displaystyle T=\frac{1}{48\pi}\frac{qr_{H}}{4}\Box\Phi~{},$ (15)

corresponding to the stress tensor (14), the theory is not conformal. However,
within the small vicinity of the horizon, we will show below that the theory
becomes conformal. In the ($t,r^{*}$) coordinate, equation of motion for
$\Phi$ (13) under the background (1) reads,

$\displaystyle-\partial^{2}_{t}\Phi+\partial^{2}_{r^{*}}\Phi=F\Big{(}\frac{qr_{H}}{16}+\frac{2}{qr_{H}}\Big{)}^{-1}R~{}.$
(16)

Now for the metric (1), $R$ is non-zero and since $F\rightarrow 0$ near the
horizon, right hand side of the above is negligible. Therefore, (16) reduces
to

$\displaystyle\partial^{2}_{t}\Phi-\partial^{2}_{r^{*}}\Phi=0~{},$ (17)

which signifies that due to existence of the horizon, the free scalar field is
propagating in flat spacetime near the horizon. Hence the trace (15) vanishes
and it implies that the Poisson algebra of the components of trace tensor (14)
closes and they form the Virasoro algebra very near to the horizon. Therefore,
the theory of the scalar field $\Phi$, whose tress tensor is given by (14), is
conformal when one considers it in the small vicinity of the horizon.

Let us now proceed to obtain the Virasoro algebra among the charges
corresponding to the near horizon diffeomorphism symmetry generators. Since
near the horizon the ingoing modes contribute, only the $T_{vv}$ component of
(14) is important. Under the metric (3), $T_{vv}$ component comes out to be

$\displaystyle
T_{vv}=\frac{1}{48\pi}\Big{[}\frac{1}{2}\partial_{v}\Phi\partial_{v}\Phi-\frac{qr_{H}}{4}\Big{(}\partial_{v}^{2}\Phi-\frac{\partial_{v}F(r)}{F(r)}\partial_{v}\Phi\Big{)}\Big{]}~{}.$
(18)

Again, since we are interested only near the horizon where
$\frac{\partial_{v}F(r)}{F(r)}\rightarrow\frac{F^{\prime}(r_{H})}{2}=\kappa$,
the above reduces to the following form,

$\displaystyle
T_{vv}=\frac{1}{48\pi}\Big{[}\frac{1}{2}\partial_{v}\Phi\partial_{v}\Phi-\frac{qr_{H}}{4}\partial_{v}^{2}\Phi+\frac{qr_{H}\kappa}{4}\partial_{v}\Phi\Big{]}~{}.$
(19)

Here, we shall determine the entropy by counting the horizon microstates of
(1) via Cardy formula [22],

$\displaystyle S=2\pi\sqrt{\frac{CL_{0}}{6}}$ (20)

corresponding to the quantum Virasoro algebra

$\displaystyle[L_{n},L_{m}]=(n-m)L_{n+m}+\frac{C}{12}n(n^{2}-1)\delta_{n+m,0}~{}.$
(21)

In the above, $[,]$ is the commutator while $C$ and $L_{n}$ are the central
charge and the conserved charge for $n$th mode respectively. For the present
case we define $L_{n}$ as,

$\displaystyle L_{n}=\int dv\xi_{n}(v)T_{vv}(v)$ (22)

where $\xi_{n}(v)$ is the near horizon diffeomophism symmetry generator,
obeying one sub-algebra isomorphic to Diff $S^{1}$,

$\displaystyle[\xi_{n},\xi_{m}]=(n-m)\xi_{n+m}~{}.$ (23)

Following [24], the explicit form of these generators for the metric (1)
satisfying (23) can be obtained. These are given by,

$\displaystyle\xi_{n}=\frac{1}{\kappa}e^{in\kappa v}$ (24)

where $n$ is an integer, can be positive and negative as well. The coordinate
$v$ is periodic within the range $0\leq v\leq\frac{2\pi}{\kappa}$ and hence we
choose this as the limits of the integration (22).

Now, in order to obtain the commutator of the conserved charges (22), let us
first calculate the Poisson bracket $\\{L_{n},L_{m}\\}$. This will contain a
Poisson bracket $\\{T_{vv}(v),T_{vv}(\bar{v})\\}$. To evaluate the bracket, we
need the following basic Poisson bracket relation

$\displaystyle\\{\Phi(r^{*},t),\partial_{t}\Phi(\bar{r^{*}},t)\\}=48\pi\delta(r^{*}-\bar{r^{*}})$
(25)

and so consequently, we have,

$\displaystyle\\{(\partial_{t}+\partial_{r^{*}})\Phi(r^{*},t),(\partial_{t}+\partial_{\bar{r^{*}}})\partial_{t}\Phi(\bar{r^{*}},t)\\}=96\pi\partial_{r^{*}}\delta(r^{*}-\bar{r^{*}})~{}.$
(26)

The bracket (25) is actually the bracket among the field and its conjugate
momentum. The normalization in the right hand side has been fixed by the
normalization in the corresponding action. In radial null coordinates, above
bracket reduces to the following form,

$\displaystyle\\{\partial_{v}\Phi(v),\partial_{\bar{v}}\Phi(\bar{v})\\}=48\pi\partial_{v}\delta(v-\bar{v})~{}.$
(27)

For the present case $T_{vv}(v)$ is given by (19). So using (27) we obtain

$\displaystyle\\{T_{vv}(v),T_{vv}(\bar{v})\\}$ $\displaystyle=$
$\displaystyle\frac{1}{48\pi}\Big{[}\partial_{v}\Phi(v)\partial_{\bar{v}}\Phi(\bar{v})\partial_{v}\delta(v-\bar{v})-\frac{qr_{H}}{4}\partial_{v}\Phi(v)\partial_{v}\partial_{\bar{v}}\delta(v-\bar{v})$
(28) $\displaystyle+$
$\displaystyle\frac{qr_{H}\kappa}{4}\partial_{v}\Phi(v)\partial_{v}\delta(v-\bar{v})-\frac{qr_{H}}{4}\partial_{\bar{v}}\Phi(\bar{v})\partial_{v}^{2}\delta(v-\bar{v})$
$\displaystyle+$
$\displaystyle\Big{(}\frac{qr_{H}}{4}\Big{)}^{2}\partial_{\bar{v}}\partial_{v}^{2}\delta(v-\bar{v})-\Big{(}\frac{qr_{H}}{4}\Big{)}^{2}\kappa\partial_{v}^{2}\delta(v-\bar{v})$
$\displaystyle+$
$\displaystyle\frac{qr_{H}\kappa}{4}\partial_{\bar{v}}\Phi(\bar{v})\partial_{v}\delta(v-\bar{v})-\Big{(}\frac{qr_{H}}{4}\Big{)}^{2}\kappa\partial_{v}\partial_{\bar{v}}\delta(v-\bar{v})$
$\displaystyle+$
$\displaystyle\Big{(}\frac{qr_{H}}{4}\Big{)}^{2}\kappa^{2}\partial_{v}\delta(v-\bar{v})\Big{]}~{}.$

Hence, considering that the auxiliary field $\Phi$ and its derivatives with
respect to $v$ are periodic within the interval $0\leq v\leq 2\pi/\kappa$, we
find,

$\displaystyle\\{L_{n},L_{m}\\}$ $\displaystyle=$ $\displaystyle\int
dv\Big{(}\xi_{n}\partial_{v}\xi_{m}-\xi_{m}\partial_{v}\xi_{n}\Big{)}T_{vv}-\frac{1}{48\pi}\Big{(}\frac{qr_{H}\kappa}{4}\Big{)}^{2}\int
dv\xi_{m}\partial_{v}\xi_{n}$ (29) $\displaystyle-$
$\displaystyle\frac{1}{48\pi}\Big{(}\frac{qr_{H}}{4}\Big{)}^{2}\int
dv\partial_{v}\xi_{m}\partial_{v}^{2}\xi_{n}~{}.$

Finally, substituting the expressions for $\xi_{n}$ from (24) in the above and
performing the integration we obtain,

$\displaystyle
i\\{L_{n},L_{m}\\}=(n-m)L_{n+m}+\frac{1}{24}\Big{(}\frac{qr_{H}}{4}\Big{)}^{2}(n^{3}+n)\delta_{n+m,0}~{}.$
(30)

Application of the usual prescription for getting the commutator relation from
the Poisson bracket yields, in the unit $\hbar=1$,

$\displaystyle[L_{n},L_{m}]=i\\{L_{n},L_{m}\\}=(n-m)L_{n+m}+\frac{1}{24}\Big{(}\frac{qr_{H}}{4}\Big{)}^{2}(n^{3}+n)\delta_{n+m,0}~{}.$
(31)

In order to put this similar to the standard form (21), let us redefine
$L_{n}$ as,

$\displaystyle L_{n}\rightarrow
L_{n}-\frac{1}{24}\Big{(}\frac{qr_{H}}{4}\Big{)}^{2}\delta_{n,0}~{}.$ (32)

This redefinition leads to the following algebra:

$\displaystyle[L_{n},L_{m}]=(n-m)L_{n+m}+\frac{1}{24}\Big{(}\frac{qr_{H}}{4}\Big{)}^{2}n(n^{2}-1)\delta_{n+m,0}~{}.$
(33)

Comparing this with (21) we obtain the central charge for the black hole case,

$\displaystyle C=\frac{q^{2}r_{H}^{2}}{32}~{}.$ (34)

Next step is to find the value of $L_{0}$. By the definition for $L_{n}$ (22)
we have,

$\displaystyle L_{0}=\frac{1}{48\pi\kappa}\int
dv\Big{[}\frac{1}{2}\partial_{v}\Phi\partial_{v}\Phi-\frac{qr_{H}}{4}\partial_{v}^{2}\Phi+\frac{qr_{H}\kappa}{4}\partial_{v}\Phi\Big{]}~{}.$
(35)

Here, since $\Phi$ and its derivative terms with respective to coordinate $v$
are periodic within the interval $0\leq v\leq 2\pi/\kappa$, the following
intregal vanishes:

$\displaystyle\int_{0}^{2\pi/\kappa}dv~{}\partial_{v}^{2}\Phi=\partial_{v}\Phi|_{0}^{2\pi/\kappa}=0,$
$\displaystyle\int_{0}^{2\pi/\kappa}dv~{}\partial_{v}\Phi=\Phi|_{0}^{2\pi/\kappa}=0~{}.$
(36)

To evaluate the first integral in (35) we will use the following trick. Let us
first find the solution of $\Phi$ from (13). For simplicity we consider the
Schwarzschild black hole. Also, our region of interest is near the event
horizon and therefore we need only ($t-r$)-sector of the metric. In this case,
the solution is given by [45],

$\displaystyle\Phi=\Big{(}\frac{qr_{H}}{16}+\frac{2}{qr_{H}}\Big{)}^{-1}\Big{[}at-\ln\Big{(}1-\frac{2M}{r}\Big{)}+A\Big{\\{}r+2M\ln\Big{(}r-2M\Big{)}\Big{\\}}+B\Big{]}~{},$
(37)

where $a$, $A$ and $B$ are arbitrary constants. Now it has been shown that the
relevant vacuum state to obtain the Hawking flux, in the context of
gravitational anomalies (both in vector case [45] and in chiral case [17]), is
the Unruh vacuum. In this vacuum, the solution for $\Phi$ near the horizon
comes out to be [45],

$\displaystyle\Phi\approx-\Big{(}\frac{qr_{H}}{16}+\frac{2}{qr_{H}}\Big{)}^{-1}\frac{v}{4M}+\textrm{constant}~{}.$
(38)

Using this value we obtain,

$\displaystyle\int_{0}^{2\pi/\kappa}dv~{}(\partial_{v}\Phi)^{2}$
$\displaystyle=$
$\displaystyle-\int_{0}^{2\pi/\kappa}dv~{}\Phi\partial^{2}\Phi\approx\Big{(}\frac{qr_{H}}{16}+\frac{2}{qr_{H}}\Big{)}^{-1}\frac{1}{4M}\int_{0}^{2\pi/\kappa}dv~{}v\partial^{2}_{v}\Phi$
(39) $\displaystyle=$
$\displaystyle-\Big{(}\frac{qr_{H}}{16}+\frac{2}{qr_{H}}\Big{)}^{-1}\frac{1}{4M}\int_{0}^{2\pi/\kappa}dv~{}\partial_{v}\Phi=0~{},$

and so $L_{0}$ vanishes. Therefore, the redefined $L_{n}$ (32) yields,

$\displaystyle L_{0}=\frac{q^{2}r_{H}^{2}}{384}$ (40)

Now, substituting (34) and (40) in the Cardy formula (20), we obtain the
entropy,

$\displaystyle S=\frac{q^{2}l_{p}^{2}}{96\sqrt{2}}S_{BH}$ (41)

where $S_{BH}=\frac{\pi r_{H}^{2}}{l_{p}^{2}}$ is the Bekenstein-Hawking
entropy and $l_{p}$ is the Planck length. Note that the conformal entropy $S$
is equal to the Bekenstein-Hawking entropy if we choose
$q^{2}=\frac{96\sqrt{2}}{l_{p}^{2}}$.

Now several comments are in order. (i) To see that there is no dimensional
ambiguity in the transformations (11), remember, in two dimensions $\Phi$ is
dimensionless if we choose Lorentz-Heaviside unit. Hence $qr_{H}$ must be
dimensionless and this is actually the case since the dimension of $q$, as we
just observed, is inverse of length. (ii) Here we obtained the entropy from
the anomalies which are the result of the quantization of the matter fields
near the horizon. So the entropy (41) is not the entropy of the black hole,
rather it can be interpreted as that of the matter. But interestingly, the
matter entropy is proportional to the horizon area. Similar has been observed
earlier in [46] for a freely falling box of ideal gas in a black hole. (iii)
Both the central charge $C$ and the zero mode eigenvalue $L_{0}$ depend on
$q$. The dependence of $C$ on $q$ is similar to [21] while that of $L_{0}$ on
$q$ is different. This is because the we applied a different method to find
$L_{0}$. Solodukhin determined it from a more general zero mode solution of
field equation near the horizon and imposing specific boundary conditions.
While we found it by just redefining $L_{n}$ and hence in our case $L_{0}$ is
proportional to $q^{2}$. Therefore the final expression for entropy contains
the parameter $q$. This ambiguity is because a direct evaluation leads to
vanishing $L_{0}$ for the classical configuration. Depending on the
regularization of the ground state one can obtain a non-zero value and
different regularization leads to different value. Of course, one might be
interested to see if there exists any method in which $L_{0}$ will come out to
be inversely proportional to $q^{2}$ and in that case the expression for
entropy will not contain any undetermined coefficient.

##  4 Chiral theory, Virasoro algebra and entropy

In chiral theory, if one of the symmetries (conformal or diffeomorphism)
breaks down then naturally other must breaks down. Hence both trace and
diffeomorphism anomalies will appear in this theory. The analogous energy-
momentum tensor is given by [47, 16, 17],

$\displaystyle\tilde{T}_{ab}=\frac{1}{48\pi}\Big{[}\frac{1}{4}\tilde{D}_{a}\tilde{G}\tilde{D}_{b}\tilde{G}-\frac{1}{2}\tilde{D}_{a}\tilde{D}_{b}\tilde{G}+\frac{1}{2}\tilde{g}_{ab}\tilde{R}\Big{]}$
(42)

where
$\tilde{D}_{a}=\tilde{\nabla}_{a}\pm\tilde{\bar{\epsilon}}_{ab}\tilde{\nabla}^{b}$
is the chiral derivative and the auxiliary field $\tilde{G}$ satisfies the
equation of motion,

$\displaystyle\tilde{\Box}\tilde{G}=\tilde{R}~{}.$ (43)

Here $+$ ($-$) corresponds to the ingoing (outgoing) mode. Since, again the
near horizon theory will be considered where only ingoing mode exists, we will
concentrate on the plus sign. This stress tensor corresponds to both the trace
and diffeomorphism anomalies which are in covariant form. The expressions are:

$\displaystyle\tilde{T}^{a}_{a}=\frac{\tilde{R}}{48\pi};\,\,\,\
\tilde{\nabla}^{a}\tilde{T}_{ab}=-\frac{1}{96\pi}\tilde{\bar{\epsilon}}_{bc}\tilde{\nabla}^{c}\tilde{R}~{}.$
(44)

Now, as earlier, to get a form of energy-momentum tensor similar to that of
Liouville theory the transformations (11) will be applied on (42) and (43).
Then proceeding in the identical way we obtain the following transformed
energy-momentum tensor,

$\displaystyle
T_{ab}=\frac{1}{48\pi}\Big{[}\frac{1}{8}D_{a}GD_{b}G-\frac{qr_{H}}{16}D_{a}D_{b}G+\frac{1}{4}g_{ab}\Big{(}\frac{qr_{H}}{4}\Box
G-\frac{1}{4}\nabla_{c}G\nabla^{c}G+\frac{1}{2}(\frac{qr_{H}}{8})^{2}\nabla_{c}G\nabla^{c}G\Big{)}\Big{]}~{}.$
(45)

Near the horizon, since only ingoing modes will be considered, under the
background metric (3) we must have,

$\displaystyle D_{v}=2\nabla_{v};\,\,\,\ D_{u}=0~{}.$ (46)

Therefore, $T_{vv}$ component is given by (18). Hence following the identical
steps we find the entropy as (41).

##  5 Conclusions

The effective theory near the event horizon of black hole corresponds to
($1+1$)-dimensional metric and is conformal. Such situation gives rise to the
two dimensional gravitational anomaly at the quantum level. Depending on the
nature of the theory (vector or chiral), there exits trace [7] or both trace
and diffeomorphism anomalies [8, 9, 10]. It has been shown that the Hawking
flux can be derived from these anomaly expressions [11, 12, 13, 15]. So it may
be possible that the anomalies can play a crucial role to illuminate the
quantum nature of a black hole. Hence, one has to see if they have a wide
applicability in the different aspects of the gravity.

In this paper, we made an attempt to the thermodynamics of the black hole
system. Here we found the entropy from the anomalous energy-momentum tensor
corresponding to the particular anomaly/anomalies. We discussed both vector as
well as chiral theories. We adopted the conformal field theory technique
proposed earlier in [5, 6, 21]. First, the expressions for energy-momentum
tensors were casted to that of Liouville theory, since this a relevant
conformal theory. Then the conserved charges corresponding to the near horizon
ingoing tensor and diffeomorphism symmetry generators were defined. An
explicit calculation of commutator between these charges gave rise to the
Virasoro algebra which had the central extension. Finally, identifying the
central charge and calculating the charge of the zero mode, we derived the
entropy by the Cardy formula.

It was noted that the entropy came out to be proportional to the well known
Bekenstein-Hawking expression. The proportionality constant contained a
undetermined parameter “$q$”, originally appeared in the transformations (11).
To obtain the exact expression one needed to choose a particular value of it.
The similar situation has been encountered earlier in [38, 40]. The
independent derivation of such choice may be interesting and will make this
method more appealing. Unfortunately, it is not known to the author.

Despite this limitation, our paper addressed a glaring omission in the
literature. A way to find the entropy from the gravitational anomalies has
been given. This may explore the importance of anomalies in studying the
quantum nature of gravity.

## References

  * [1] S. W. Hawking, Nature 248, 30 (1974).
  * [2] J. D. Bekenstein, Phys. Rev. D 7, 2333 (1973).
  * [3] A. Ashtekar, J. Baez, A. Corichi and K. Krasnov, Phys. Rev. Lett. 80, 904 (1998) [arXiv:gr-qc/9710007].
J. M. Garcia-Islas, Class. Quant. Grav. 25, 245001 (2008) [arXiv:0804.2082
[gr-qc]].

  * [4] A. Strominger and C. Vafa, Phys. Lett. B 379, 99 (1996) [arXiv:hep-th/9601029].
  * [5] J. D. Brown and M. Henneaux, Commun. Math. Phys. 104, 207 (1986).
  * [6] A. Strominger, JHEP 9802, 009 (1998) [arXiv:hep-th/9712251].
  * [7] P. C. W. Davies and S. A. Fulling, Proc. R. Soc. Lond. A 354, 59 (1977).
  * [8] L. Alvarez-Gaume, E. Witten, Nucl. Phys. B234, 269 (1984).
  * [9] W. A. Bardeen and B. Zumino, Nucl. Phys. B 244, 421 (1984).
  * [10] R. A. Bertlmann and E. Kohlprath, Annals Phys. 288, 137 (2001) [arXiv:hep-th/0011067].
  * [11] S. M. Christensen and S. A. Fulling, Phys. Rev. D 15, 2088 (1977).
  * [12] S. P. Robinson and F. Wilczek, Phys. Rev. Lett. 95, 011303 (2005) [arXiv:gr-qc/0502074].
  * [13] S. Iso, H. Umetsu and F. Wilczek, Phys. Rev. Lett. 96, 151302 (2006) [arXiv:hep-th/0602146].
  * [14] K. Umetsu, Int. J. Mod. Phys. A 25, 4123 (2010) [arXiv:0907.1420 [hep-th]].
  * [15] R. Banerjee and S. Kulkarni, Phys. Rev. D 77, 024018 (2008) [arXiv:0707.2449 [hep-th]].
  * [16] R. Banerjee and S. Kulkarni, Phys. Lett. B 659, 827 (2008) [arXiv:0709.3916 [hep-th]].
  * [17] R. Banerjee and S. Kulkarni, Phys. Rev. D 79, 084035 (2009) [arXiv:0810.5683 [hep-th]].
  * [18] R. Banerjee and B. R. Majhi, Phys. Rev. D 79, 064024 (2009) [arXiv:0812.0497 [hep-th]].
  * [19] V. Akhmedova, T. Pilling, A. de Gill and D. Singleton, Phys. Lett. B 673, 227 (2009) [arXiv:0808.3413 [hep-th]].
A. Zampeli, D. Singleton and E. C. Vagenas, JHEP 1206, 097 (2012)
[arXiv:1206.0879 [gr-qc]].

  * [20] O. Coussaert, M. Henneaux and P. van Driel, Class. Quant. Grav. 12, 2961 (1995) [arXiv:gr-qc/9506019].
  * [21] S. N. Solodukhin, Phys. Lett. B 454, 213 (1999) [arXiv:hep-th/9812056].
  * [22] J. L. Cardy, Nucl. Phys. B 270 (1986) 186.
H. W. J. Bloete, J. L. Cardy and M. P. Nightingale, Phys. Rev. Lett. 56, 742
(1986).

  * [23] S. Carlip, Phys. Rev. Lett. 82, 2828 (1999) [arXiv:hep-th/9812013].
  * [24] S. Carlip, Class. Quant. Grav. 16, 3327 (1999) [arXiv:gr-qc/9906126].
  * [25] F. L. Lin and Y. S. Wu, Phys. Lett. B 453, 222 (1999) [arXiv:hep-th/9901147].
V. O. Solovev, Phys. Rev. D 61, 027502 (2000) [arXiv:hep-th/9905220].
M. Natsuume, T. Okamura and M. Sato, Phys. Rev. D 61, 104005 (2000)
[arXiv:hep-th/9910105].
D. J. Navarro, J. Navarro-Salas and P. Navarro, Nucl. Phys. B 580, 311 (2000)
[arXiv:hep-th/9911091].
J. l. Jing and M. L. Yan, Phys. Rev. D 62, 104013 (2000) [arXiv:gr-
qc/0004061].

  * [26] S. Das, A. Ghosh and P. Mitra, Phys. Rev. D 63, 024023 (2001) [arXiv:hep-th/0005108].
  * [27] J. l. Jing and M. L. Yan, Phys. Rev. D 63, 024003 (2001) [arXiv:gr-qc/0005105].
H. Terashima, Phys. Lett. B 499, 229 (2001) [arXiv:hep-th/0011010].
M. Hotta, K. Sasaki and T. Sasaki, Class. Quant. Grav. 18, 1823 (2001)
[arXiv:gr-qc/0011043].

  * [28] H. Terashima, Phys. Rev. D 64, 064016 (2001) [arXiv:hep-th/0102097].
  * [29] J. i. Koga, Phys. Rev. D 64, 124012 (2001) [arXiv:gr-qc/0107096].
  * [30] M. I. Park, Nucl. Phys. B 634, 339 (2002) [arXiv:hep-th/0111224].
  * [31] S. Carlip, Phys. Rev. Lett. 88, 241301 (2002) [arXiv:gr-qc/0203001].
  * [32] M. Cvitan, S. Pallua and P. Prester, Phys. Lett. B 555, 248 (2003) [arXiv:hep-th/0212029].
A. J. M. Medved, D. Martin and M. Visser, Class. Quant. Grav. 21, 3111 (2004)
[arXiv:gr-qc/0402069].
G. Kang, J. i. Koga and M. I. Park, Phys. Rev. D 70, 024005 (2004) [arXiv:hep-
th/0402113].
A. Giacomini, Phys. Rev. D 70, 044005 (2004) [arXiv:hep-th/0403219].
M. Cvitan, S. Pallua and P. Prester, Phys. Rev. D 70, 084043 (2004)
[arXiv:hep-th/0406186].
S. Carlip, Class. Quant. Grav. 22, 1303 (2005) [arXiv:hep-th/0408123].
M. Cvitan and S. Pallua, Phys. Rev. D 71, 104032 (2005) [arXiv:hep-
th/0412180].
S. Carlip, Class. Quant. Grav. 22, R85 (2005) [arXiv:gr-qc/0503022].
S. Carlip, Phys. Rev. Lett. 99, 021301 (2007) [arXiv:gr-qc/0702107].
S. Carlip, Gen. Rel. Grav. 39, 1519 (2007) [Int. J. Mod. Phys. D 17, 659
(2008)] [arXiv:0705.3024 [gr-qc]].
L. M. Cao, Y. Matsuo, T. Tsukioka and C. M. Yoo, Phys. Lett. B 679, 390 (2009)
[arXiv:0906.2267 [hep-th]].
L. Rodriguez and T. Yildirim, Class. Quant. Grav. 27, 155003 (2010)
[arXiv:1003.0026 [hep-th]].
B. K. Button, L. Rodriguez and C. A. Whiting, arXiv:1009.1661 [hep-th].
H. Chung, Phys. Rev. D 83, 084017 (2011) [arXiv:1011.0623 [gr-qc]].
A. Guneratne, L. Rodriguez, S. Wickramasekara and T. Yildirim, arXiv:1206.2261
[hep-th].

  * [33] M. I. Park and J. Ho, Phys. Rev. Lett. 83, 5595 (1999) [arXiv:hep-th/9910158].
S. Carlip, Phys. Rev. Lett. 83, 5596 (1999) [arXiv:hep-th/9910247].

  * [34] M. I. Park and J. H. Yee, Phys. Rev. D 61, 088501 (2000) [arXiv:hep-th/9910213].
  * [35] O. Dreyer, A. Ghosh and J. Wisniewski, Class. Quant. Grav. 18, 1929 (2001) [arXiv:hep-th/0101117].
  * [36] S. Silva, Class. Quant. Grav. 19, 3947 (2002) [arXiv:hep-th/0204179].
  * [37] R. Brustein, Phys. Rev. Lett. 86, 576 (2001) [arXiv:hep-th/0005266].
  * [38] M. Cvitan, S. Pallua and P. Prester, Phys. Lett. B 546, 119 (2002) [arXiv:hep-th/0207265].
  * [39] A. Giacomini and N. Pinamonti, JHEP 0302, 014 (2003) [arXiv:gr-qc/0301038].
N. Pinamonti and L. Vanzo, Phys. Rev. D 69, 084012 (2004) [arXiv:hep-
th/0312065].

  * [40] G. A. S. Dias and J. P. S. Lemos, Phys. Rev. D 74, 044024 (2006) [arXiv:hep-th/0602144].
  * [41] B. R. Majhi and T. Padmanabhan, Phys. Rev. D 85, 084040 (2012) [arXiv:1111.1809 [gr-qc]].
B. R. Majhi and T. Padmanabhan, arXiv:1204.1422 [gr-qc] (to appear in Phys.
Rev. D (Rapid Comm.)).

  * [42] J. G. Russo, Phys. Lett. B 359, 69 (1995) [hep-th/9507009].
  * [43] S. Deser, M. J. Duff and C. J. Isham, Nucl. Phys. B 111, 45 (1976).
M. J. Duff, Nucl. Phys. B 125, 334 (1977).

  * [44] A. M. Polyakov, Phys. Lett. B 103, 207 (1981).
  * [45] R. Balbinot, A. Fabbri and I. L. Shapiro, Nucl. Phys. B 559, 301 (1999) [arXiv:hep-th/9904162].
  * [46] S. Kolekar and T. Padmanabhan, Phys. Rev. D 83, 064034 (2011) [arXiv:1012.5421 [gr-qc]].
  * [47] H. Leutwyler, Phys. Lett. B 153, 65 (1985) [Erratum-ibid. 155B, 469 (1985)].


# The Impact of Visual Appearance on User Response in Online Display
Advertising

Javad Azimi
<EMAIL_ADDRESS>
Oregon State University  Ruofei Zhang
rzhang @yahoo-inc.com
Yahoo! Labs, Silicon Valley  Yang Zhou
yangzhou @yahoo-inc.com
Yahoo! Labs, Silicon Valley  Vidhya Navalpakkam
<EMAIL_ADDRESS>
Yahoo! Labs, Silicon Valley  Jianchang Mao
<EMAIL_ADDRESS>
Yahoo! Labs, Silicon Valley  Xiaoli Fern
<EMAIL_ADDRESS>
Oregon State University

###### Abstract

Display advertising has been a significant source of revenue for publishers
and ad networks in online advertising ecosystem. One of the main goals in
display advertising is to maximize user response rate for advertising
campaigns, such as click through rates (CTR) or conversion rates. Although in
the online advertising industry we believe that the visual appearance of ads
(creatives) matters for propensity of user response, there is no published
work so far to address this topic via a systematic data-driven approach. In
this paper we quantitatively study the relationship between the visual
appearance and performance of creatives using large scale data in the world’s
largest display ads exchange system, RightMedia. We designed a set of $43$
visual features, some of which are novel and some are inspired by related
work. We extracted these features from real creatives served on RightMedia. We
also designed and conducted a series of experiments to evaluate the
effectiveness of visual features for CTR prediction, ranking and performance
classification. Based on the evaluation results, we selected a subset of
features that have the most important impact on CTR. We believe that the
findings presented in this paper will be very useful for the online
advertising industry in designing high-performance creatives. It also provides
the research community with the first ever data set, initial insights into
visual appearance’s effect on user response propensity, and evaluation
benchmarks for further study.

##  1 Introduction

The Internet revolution has transformed how people experience information,
media and advertising. Web advertising, although nonexisting twenty years ago,
has become a vital component of the modern Internet, where advertisements are
delivered from advertisers to users through different online channels. Recent
trends have shown that an increasingly large share of advertisers’ budgets are
devoted to the online world, and online advertising spending has greatly
outpaced some of the traditional advertising media, such as radio and
magazine. Display advertising is one type of online advertising which,
together with search advertising, contributes the majority of the revenue for
many large Internet companies. In display advertising, display ad instances
are shown to the user on webpages in different formats such as image, flash,
and video. Each display ad instance is called a creative. By showing the
creatives, advertisers aim to either promote brand awareness among users
(brand advertising) or receive desirable responses from users (performance
advertising), such as the action of purchasing, clicking or signing up for a
promotion list from the advertiser’s website. In performance advertising, the
advertiser strives to optimize their ad’s performance metrics such as the
effective cost per click (eCPC) or effective cost per action (eCPA), which in
turn relates to maximizing the user response rate on the creatives as measured
by click through rates (CTR) or conversion rates (CVR). There are several
factors that greatly influence the user response rate of display advertising
campaigns: 1) the position of the ads on the webpage; 2) the relevancy of the
ads to the online users, which is generally captured by the targeting profiles
of the advertising campaigns; 3) the relevancy of the ads to the webpage
content and 4) the quality and visual appearance of the creatives.

The problem of predicting the user response rate for online ads, especially
CTR, has been studied by several researchers in the last few years. One major
research focus has been in predicting clicks by studying the relationship
between CTR and the aforementioned ad factors (and their combinations). For
example in [2], the authors considered the ad’s relevancy to the content of
the webpage in predicting CTR. They show that improving the ad’s content
relevancy is more efficient than considering the content of ads by themselves
[25]. Although it is generally believed that visually appealing ads can
perform better in attracting online users, as a result of which advertisers
always care about the creative designs, there is no, to the best of our
knowledge, published work so far to quantitatively study the effect of visual
appearance of creatives on campaign performance in online display advertising.
This motivates us to investigate the correlation between the visual features
of the creative and CTR, regardless of other ad factors, and to predict
creative performance based on its visual appearance alone.

Our proposed approach consists of two main steps, 1) feature extraction and 2)
correlation investigation. We first extract some informative visual features
from the creatives. We introduce $43$ visual features classified into three
categories, 1) _global features_ which characterize the overall properties of
a given creative, 2) _local features_ representing the properties of specific
parts within a given creative and 3) _advanced features_ which are a group of
features developed based on more complicated algorithms such as the number of
faces and number of characters in a creative. We then develop three regression
approaches to predict the CTR based on these features. The study is conducted
using real creatives and their performance data from the world’s largest
display ads exchange system, RightMedia. Based on the weights of developed
features, we further select a subset of features that have high impact on the
creative’s CTR. The benefit of this work is three-fold. First, our findings on
the visual features and their relationship to CTR can provide useful
recommendations to designers on what features to consider while designing
creatives, and/or can help in automated creative generation. Second, the
visual features and the regression methods developed here can be used in
addition to the traditionally investigated ad factors (such as ad relevancy,
position etc.) for improving CTR prediction in online ads selection. Third, it
provides the research community with the first ever data set, initial insights
into the effect of visual appearance on user response propensity, and
evaluation benchmarks for further study.

The paper is organized as follows. Section 2 introduces the related work. We
introduce the visual features in Section 3. The regression and feature
selection results for CTR prediction are presented in Section 4, followed by
our conclusion in Section 5.

##  2 Background and Related Work

The relationship between various print ad characteristics and measures of
advertising effectiveness has been studied by advertising researchers for
almost a century. A wide variety of characteristics have been investigated.
These characteristics are roughly in two categories: mechanical and content-
based. The mechanical characteristics include ad size, number of colors,
proportional of illustrations to copy, the absence of borders, and type size.
The content factors include message appeal like status, quality, fear and
fantasy, attention-getting techniques like free offers, presence of women, and
psycholinguistic variables like product or personal reference in headline,
interrogative or imperative headline, visual rhetorics, among others. See [20]
for summaries.

Even though the online advertising has taken a large market share of the
advertising industry, and the whole industry is steadily and continuously
shifting to the online domain, study on the effectiveness of the counterpart
of print ads online, generally called display ads, is limited. We list the
studies of several factors below.

Some existing studies try to investigate the effect of several different
factors on the performance of display advertising campaigns. These factors
include targeting and obtrusiveness [9], advertisement size (large vs. small)
and ad exposure format (intrusive vs. voluntary) [4], cognitive impact from ad
size and animation [18], emotional appeal and incentive offering in the ads
[8], repetition of varied execution vs. single execution [30].

To the best of our knowledge, we are not aware of any study on the
relationship between the visual appearance and the performance of creatives in
online display ads. We try to tackle this problem by first defining a set of
visual features and then evaluating their effects on ad performance,
specifically CTR in our experiments, from the actively served ad campaigns on
the world’s largest ad exchange system, RightMedia. Below, we present some
previous computational studies on image properties which provide us
inspiration in designing our visual features.

There are several studies that try to investigate a specific property of
images (photos or paintings) using computational approaches. Such properties
include quality and aesthetic in photos [19, 15, 29, 7] or in paintings [17],
saliency [14], composition [10, 24], color harmony [5] and memorability [13].

Initial work on image quality evaluation concentrated on evaluating and
reconstructing low graded, compressed or degraded images by simple noise model
[6, 1]. However, in most of the beauty evaluation work, including this paper,
we assume that high quality images are available and we are interested in
evaluating the visual aesthetic of images based on visual features.

Recently some researchers tried to evaluate the beauty of an image based on
its visual features. In [15] the authors aim to classify the pictures into
professional and snapshot photos using some basic features including spatial
distribution of edges, color distribution and hue count, etc. In [7] the
authors introduced a regression based approach for rating photos based on
their beauty, using features such as average pixel density, colorfulness,
saturation hue, and the rule of thirds. In addition to these studies, in [19]
the author proposed an approach to classifying images into high and low
quality. The main idea comes from the fact that a professional photographer
makes the background blurry and the subject distinguishable in the image. By
separating the blurry part of the image from the subject, they design a set of
well-motivated features from both the subject and the whole image such as the
clarity contrast of the subject, lighting, simplicity, color harmony and
composition geometry. They show that the combination of these features can
provide a promising performance. All of the above work tries to extract visual
features from photos. Recently Li et al. [17] tried to extract some features
from paintings to evaluate their beauty and classify them into high and low
quality. They introduced a set of global and local features, $40$ in total, to
capture the painting properties such as the brightness contrast between
segments, the brightness contrast across the whole image and the average
saturation for the largest segment of the image.

Computational approaches have also been used to investigate other visual
properties of an image. In [13] the authors studied what properties of images
make them more memorable. They found that statistical properties of an image
such as mean hue, mean saturation, intensity mean, intensity variance,
intensity skewness and number of objects do not have any non-trivial
correlation with memorability in their generated data set. However, they found
that if they label the objects and scenes in the images, they can find a non-
trivial and interesting correlation between images and their memorability. For
example, their results show that the attendance of human being, close up
objects and human scale objects in an image improve its memorability more than
natural scene. This result is not possible to be applied to our work since it
requires large amounts of supervision to tag different parts of the images.
However, we evaluate the impact of the number of human faces in an image in
our work.

Color harmonization is another approach for making an image more appealing. In
[5] the authors proposed to harmonize the colors in a given image using
harmonization templates from [31, 28], which include $8$ different harmonized
color templates. We also used color harmony models to evaluate the hue
distribution of an image in our experiments.

In summary, existing work in related areas has focused primarily on properties
of an image, photo or painting. In contrast, we examine creatives in online
display ads, which contain both graphical features and text. In addition, some
of the existing approaches require significant amount of supervision in their
feature extraction step, which is not possible in large scale applications
where we need to learn from large data sets with minimum amount of
supervision. Finally, we would also like to extract a set of features that are
visually understandable and can be practically controlled to guide the human
designers or automatic creative generators (like in smart ads) to produce
high-performance creatives. These objectives make our problem novel and
interesting for the online advertising industry.

##  3 Feature Extraction

In this section we introduce a set of $43$ different visual features. We
categorize the developed features into three different sets, 1) global
features, 2) local features and 3) advanced features. A complete list of the
features can be found in Table 3. Below we describe the detailed definition of
the proposed features in each category.

In the following sections we use $I$ to indicate an image and use $|I|$ to
indicate the size of the image measured by the number of pixels. We use
variable $x$ to denote an arbitrary pixel when we do not care about its
location in the image. Otherwise we use $(i,j)$ to denote the pixel in the
$i$-th row and $j$-th column in the image.

###  3.1 Global Features

Global features are a set of features which represent the overall properties
of the whole image. We describe the details of $19$ different global features
in this section.

####  3.1.1  Gray Level Features

We describe 3 features extracted from the gray level histogram of the image,
namely the gray level contrast $f_{1}$, number of dominant gray level bins
$f_{2}$, and the standard deviation of the gray level values among all pixels
$f_{3}$.

The gray level contrast is the width of the middle $95\%$ mass in the gray
level histogram [15]. From the original gray level histogram, we prune the
extreme $2.5\%$ from the 0 side and $2.5\%$ from the 255 side. Gray level
contrast feature $f_{1}$ is calculated as the width of the remaining
histogram.

We count the number of dominant bins in the gray level histogram as our second
feature. Suppose the set $G=\\{g_{0},g_{1},\cdots,g_{255}\\}$ indicates the
set of $256$ bins in the gray level histogram such that $g_{i}$ is the number
of pixel in $i$-th bins. We define the number of dominant gray level bins as
$f_{2}=\sum_{k=0}^{255}\textbf{1}(g_{k}\geq c_{1}\max_{i}g_{i})$, where
$\textbf{1}(\cdot)$ is the indicator function and $c_{1}$ is a threshold value
which is set to be $0.01$ in this paper. 111This parameter, and similar ones
in the rest of the paper, is set inspired by related works such as [19].

The last gray level feature, $f_{3}$, is defined as the standard deviation of
gray level values of all pixels in the image. It is used to capture the
variance of the gray level distribution.

####  3.1.2 Color Distribution

To avoid distraction from objects in the background, professional
photographers tend to keep the background simple. In [19], the authors use the
color distribution of the background to measure this simplicity. We use a
similar approach to measure the simplicity of color distribution in the image.
For a given image, we quantize each RGB channel into $8$ values, creating a
histogram $H_{rgb}=\\{h_{0},h_{1},\cdots,h_{511}\\}$ of 512 bins, where
$h_{i}$ indicates the number of pixels in $i$-th bin. We define feature
$f_{4}$ to indicate the number of dominant colors as
$f_{4}=\sum_{k=0}^{512}\textbf{1}(h_{k}\geq c_{2}\max_{i}h_{i})$ where
$c_{2}=0.01$ is the threshold parameter. We also calculate the size of the
dominant bin relative to the image size as $f_{5}=\frac{\max_{i}h_{i}}{|I|}$.
This feature indicates the extent to which one of $512$ colors is dominant in
the image.

By replacing the RGB color map with HSV (Hue, Saturation, Value) color map and
using the above methods in calculating features $f_{4}$ and $f_{5}$, we obtain
two other features $f_{6}$ and $f_{7}$.

####  3.1.3 Model-Based Color Harmony

The concept of color harmony in this paper is based on $8$ different harmonic
color distributions (illustrated in Figure 1) that are based on the hue of the
HSV color wheel [31]. These distributions are called _i, V, L, I, T, Y, X, N_.
Note that each distribution can be rotated by $0\leq\alpha\leq 360$ degrees.
The specific size of color harmony distributions are set as follows: the large
sectors of types $V,Y$ and $X$ are $26\%$ of the disk ($93.6^{\circ}$); the
small sectors of types $i,L,I$ and $Y$ are $5\%$ of the disk ($18^{\circ}$);
the largest sector of type $L$ is $22\%$ of the disk ($79.2\%$); the sector of
type $T$ is $50\%$ of the disk ($180^{\circ}$). The angle between the centers
of the two sectors is $180^{\circ}$ for $I$, $X$, $Y$, and $90^{\circ}$ for
$L$.

Figure 1: Color harmony models

Let us define the set of $8$ distributions as
$\mathcal{D}=\\{d^{1},d^{2},\cdot\cdot\cdot,d^{8}\\}$. We say
$\phi(d^{i}_{\alpha},x)$ indicates the hue of the closest point in the $i$-th
distribution to $x$ after $\alpha$ degree rotation, where $x$ is any arbitrary
pixel in the image. We compute the distance between the hue distribution of
our image $I$ and the distribution $d^{i}\in\mathcal{D}$ as:

$\gamma(I,d^{i})=\mathop{\mathrm{argmin}}_{\alpha}\frac{1}{|I|}\sum_{x\in
I}\parallel hue(x)-\phi(d^{i}_{\alpha},x)\parallel\cdot sat(x),$ (1)

where $hue(x)$ and $sat(x)$ indicate the hue and saturation at pixel $x$, and
$\|\cdot\|$ denotes the arc-length distance. We are interested in the best
fitting model $d^{*}$ which has the least $\gamma(\cdot)$ value,
$d^{*}=\mathop{\mathrm{argmin}}_{d^{i}}\gamma(I,d^{i})$. We define feature
$f_{8}=\gamma(I,d^{*})$. Intuitively, it tells us how different is the hue
distribution of image $I$ from the best fitting model of color harmony.

Some models are superset of other models in Figure 1 concluding that the
$\gamma(\cdot)$ value of some smaller models are higher than some larger
models given any image $I$, e.g.
$\gamma(I,d_{i})\geq\gamma(I,d_{V})\geq\gamma(I,d_{T})$. Therefore, if an
image hue distribution fits into some small models, type $i,V,L,I$, it fits
into larger models as well. This can emphasize the color harmony property of
the images which can fit into a few models rather than just one model. We
consider this property as one potential positive property of the image. To
quantify this property, we introduce a new feature, $f_{9}$, which indicates
the average color harmony deviation from the best two fitted models given an
image $I$. In general, in addition to the deviation from the best fitted model
illustrated by feature $f_{8}$, we consider the deviation from the second best
fitted model as well, and the average of these two deviations is returned as
$f_{9}$. Clearly, for the images fitting into small color harmony models, we
will have $f_{8}$ and $f_{9}$ very close to each other. However, for the
images which fit into the largest model, we will have $f_{9}$ considerably
larger than $f_{8}$. We believe these two numerical features can represent the
color harmony property of an image appropriately.

####  3.1.4  Color Coherence

We extract a set of features based on the color coherence of pixels resulting
in connected coherent components [23]. A connected coherent component in an
image is defined as:

  * •

A set of pixels that fall into the same bin in the histogram.

  * •

For any two pixels $p_{i}$ and $p_{j}$ in a connected coherent component
$P=\\{p_{1},p_{2},\cdots,p_{m}\\}$ of $m$ pixels, there is a path of
sequential pixels, $p_{i},p_{i+1},\cdot\cdot\cdot,p_{j}$. Two sequential
pixels in a path must be one of the $8$ neighborhoods of each other.

  * •

The size of the connected coherent component is larger than a predefined
threshold $c_{4}$. In our experiment we set $c_{4}=0.01|I|$.

We denote the set of connected coherent components and their color index as
$\mathcal{P}=\\{(P_{1},h_{1}),(P_{2},h_{2}),\cdot\cdot\cdot(P_{n},h_{n})\\}$,
where $P_{i}$ is the set of pixels in the $i$-th component, and $h_{i}$ is its
corresponding color in the HSV color histogram with $512$ bins. We use
$|P_{i}|$ to denote the number of pixels in $P_{i}$. We extract the following
features based on the above definition:

  * •

$f_{10}=n$, which indicates the number of connected coherent components in the
image.

  * •

$f_{11}=\frac{\max_{i}|P_{i}|}{|I|}$, which indicates the size of the largest
component relative to the whole image.

  * •

$f_{12}=\frac{\max\limits_{j,j\neq\arg\max\limits_{i}|P_{i}|}|P_{j}|}{|I|}$,
representing the size of the second largest connected coherent component
relative to the whole image.

  * •

$f_{13}=\mathrm{rank}(h_{i}),i=\arg\max_{j}|P_{j}|$, indicating the rank of
the bin, considering the bin size in descending order, associated with the
largest connected coherent component in the image. For example, the value of
this feature is $1$ if the bin associated with the largest coherent component,
$\arg\max_{j}|P_{j}|$, is the largest bin in the color histogram as well;
$\max_{i}|h_{i}|$ where $h_{i}$ is the size of the $i-{th}$ bin in the color
histogram. This feature indicates how the colors are disperse in the image. We
expect to have value $f_{13}=1$ if the colors in the images are not very
randomly distributed. It means the pixel with the same colors are mostly
connected together.

  * •

$f_{14}=\mathrm{rank}(h_{i}),i=\arg\max_{j,j\neq\arg\max_{k}|P_{k}|}|P_{j}|$,
similar to $f_{13}$, it shows the bin rank, considering the bin size in
descending order, of the second largest connected coherent component in the
image.

####  3.1.5 Hue Distribution

In this section we introduce three features based on the hue in HSV color
space. We quantize hues in an image in a similar way as in [17] by eliminating
the pixels with saturation and value less than $0.2$. This will eliminate all
the pixels with white or black colors. Then we calculate the hue histogram of
remaining pixels with $20$ different bins, $18^{\circ}$ for each bin, which
results in $\mathcal{H}_{hue}=\\{h_{1},h_{2},\cdot\cdot\cdot,h_{20}\\}$ where
$h_{i}$ indicates the set of pixels in $i$-th bin. We then extract the
following features:

  * •

$f_{15}=\sum_{i=1}^{20}\textbf{1}(|h_{i}|\geq c_{5}|I|)$ where $c_{5}=0.01$ in
our experiments. This feature indicates the number of dominant hues in an
image.

  * •

$f_{16}=\max_{i,j}\parallel|h_{i}|-|h_{j}|\parallel$ where $|h_{i}|\geq
c_{5}|I|$, and $\|\cdot\|$ is the arc length. This feature indicates the
largest contrast between two dominant hues in the image.

  * •

$f_{17}=\mathrm{std}(\Phi)$ where $\Phi=\\{\cup_{i\in I}\;\parallel
h_{i}(i)-0\parallel\\}$ and $\parallel.\parallel$ is the arc length value.
This feature indicates the standard deviation of all pixel’s hues distance
from the origin $0$. It simply can determine how much the hue colors in an
images has been distributed from each other.

####  3.1.6 Lightness Features

We use the lightness $L$ in the HSL color space to calculate feature $f_{18}$
and $f_{19}$. In the HSL color space, $L$ value is small when the color is
white and is large when the color is black. The $L$ value in HSL color space
can be calculated as follows:

$L(x)=\frac{\max\left(r(x),g(x),b(x)\right)+\min\left(r(x),g(x),b(x)\right)}{2},$
(2)

where $r(x),g(x),b(x)$ denotes the R, G, B values of pixel $x$ in RGB color
space. We calculate two lightness features as:

  * •

$f_{18}=\frac{1}{|I|}\sum_{x\in I}L(x)$, the average lightness of pixels in
the image.

  * •

$f_{19}=\mathrm{std}(L(\cdot))$, the standard deviation of lightness of all
pixels in the image.

###  3.2 Local Features

Local features represent a set of features extracted from specific parts of
the image rather than the whole image. We apply the normalized cut
segmentation method [26] to partition the image into $5$ smaller segments. Let
$\mathcal{S}=\\{S_{1},S_{2},\cdots,S_{5}\\}$ indicate the set of $5$ different
segments where $S_{i}$ is the set of pixels in segment $i$. Note that a
segment is considered as noise and is dropped if it is smaller than $5\%$ of
the image. We develop the following features based on the segmentation result.

####  3.2.1 Segment Size

Two features are extracted from segment size as follows:

  * •

$f_{20}=\frac{\max_{i}|S_{i}|}{|I|}$, indicating the size of the largest
segment relative to the whole image.

  * •

$f_{21}=\frac{1}{|I|}\max_{i,j}\big{|}|S_{i}|-|S_{j}|\big{|}$, indicating the
contrast among the segmentation sizes of the image.

####  3.2.2 Segment Hues

Similar to section 3.1.5, we generate the hue histogram of each segment. We
define the set of hue histograms of all $5$ segments as
$\mathcal{H}^{s}_{hue}=\\{h_{1,1},h_{1,2},\cdot\cdot\cdot,h_{1,20},h_{2,1},\cdot\cdot\cdot,h_{5,20}\\}$
where $h_{i,j}$ indicates the set of pixels that fall in the $j$-th bin of
$i$-th segment. Then we extract five features to capture different hue
properties. Below we describe the formal definition of developed features:

  * •

$f_{22}=\sum_{j=1}^{20}\textbf{1}(|h_{i,j}\geq c_{6}|I|)$ where
$i=\arg\max_{i}S_{i}$ and $c_{6}=0.01$. This feature denotes the number of
image-wide dominant hues in the largest segment. In general, we would like to
have most of the image hues in the largest segment.

  * •

$f_{23}=\sum_{j=1}^{20}\textbf{1}(|h_{i,j}\geq c_{6}|S_{i}|)$ where
$i=\arg\max_{i}S_{i}$. This feature denotes the number of segment-wide
dominant hues in the largest segment.

  * •

$f_{24}=\max\limits_{i}q_{i}$ where
$q_{i}=\sum_{j=1}^{20}\textbf{1}(|h_{i,j}\geq c_{6}|S_{i}|)$ is the number of
dominant hues in $i-{th}$ segment. This feature essentially denotes the
largest number of dominant hues in one segment. We would like to have the same
value as $f_{23}$ for this feature illustrating that the largest segment has
the largest number of dominant colors.

  * •

$f_{25}=\max\limits_{i,j}|q_{i}-q_{j}|$. This feature denotes the contrast of
the number of dominant hues among the segments. We usually do not like to have
lots of different hues in one segment and a few hues in another segment in an
image. We expect to have unappealing images with large value for $f_{25}$.

  * •

$f_{26}=\max\limits_{j,k}\|h_{i,j}-h_{i,k}\|$ where $|h_{i,j}|,|h_{i,k}|\geq
c_{6}|S_{i}|$, $i=\arg\max_{i}|S_{i}|$ and $\|\cdot\|$ is the arc length
distance. This feature captures the contrast of number of pixels among the hue
bins in the largest segment. In general, we expect to have an appealing image
with one bin dominating the largest segment in addition to a few more small
bins. This makes the contrast value very large.

  * •

$f_{27}=\mathrm{std}(T(\cdot))$ where $T(i)=\max_{j,k}$,
$|h_{i,j}|,|h_{i,k}|\geq c_{6}|S_{i}|$. This feature returns the standard
deviation of contrast among the segments. If we have different hue contrasts
among different segments, this feature will achieve a significant value.

####  3.2.3 Segment Color Harmony

Two features are extracted based on the largest segment color harmony. Feature
$f_{28}$ is the minimum deviation from the best fitted color harmony model for
the largest segment, and feature $f_{29}$ is the average deviation of the best
two fitted color harmony models for the largest segment. The details of color
harmony models have been introduced in section 3.1.3.

####  3.2.4 Segment Lightness

Three segment lightness features are extracted using similar method as in
section 3.1.6:

  * •

$f_{30}$: average lightness in the largest segment.

  * •

$f_{31}$: standard deviation of average lightness among the segments.

  * •

$f_{32}$: contrast of average lightness among the segments.

###  3.3 Advanced features

In this section we develop a set of features based on more complicated
algorithms. Most of the advanced features are based on the saliency map of the
image which determines the visually salient areas in the image that are more
likely to be noticed by the humans. We also extract two additional features
related to the number of characters and number of faces in an image. Below we
describe the details of these features.

####  3.3.1 Saliency Features

Saliency computation is a well known phenomenon in human vision where
attention tends to be drawn to interesting parts of an image that appear
visually different from the rest of the image (e.g., a red coke can in a green
background appears salient and is immediately noticed, while the same coke can
in an orange-reddish background is not salient and less likely to be noticed).
We compute saliency according to the algorithm described in [12]. Figure 2
shows the saliency output of the algorithm presented in [12] for a sample
creative. The areas with higher lightness in the saliency map indicate more
salient part of the image.

Figure 2: The saliency map of an image. Left: original image. Right: saliency
map.

The saliency algorithm returns a matrix $\tau$ (also referred to as saliency
map) where $\tau(i,j)$ represents the saliency value of pixel $(i,j)$. We also
extract a binary image based on the saliency map, by setting a threshold
$\alpha$ to the saliency map where the pixels with saliency value larger than
$\alpha$ are set to $1$ and the rest of the pixels are set to $0$. Similar to
[12], the parameter $\alpha$ is set as $\alpha=3\bar{\tau}$ where
$\bar{\tau}=1/n\sum_{i,j}\tau(i,j)$ is the average saliency value in the
image. After this binarization, we have some connected components with value
1. These components indicate saliency areas, and the other parts of the image
are considered as background. Then we extract the following features based on
the saliency results, saliency map and binary saliency map.

  * •

$f_{33}$: background size. Salient objects usually appear in the foreground
and not in the background. Therefore we return the size of the background as a
function of image size which is calculated as:
$f_{33}=\frac{\sum_{i,j}\textbf{1}(\tau(i,j)<\alpha)}{|I|}.$

  * •

$f_{34}$: number of connected components in the binary map.

  * •

$f_{35}$: size of the largest components in the binary saliency map relative
to the whole image.

  * •

$f_{36}$: average saliency value of the largest component in the binary
saliency map.

  * •

$f_{37}$: number of connected components in the image background. In some
images, the saliency areas can divide the background into several disconnected
segments. Usually it is not desirable to have multiple background components.

  * •

$f_{38}$: size of the largest connected component in the background relative
to the whole image. If the number of connected components in the background is
equal to one, then this feature has the same value as $f_{33}$.

  * •

$f_{39}$: distance between connected components. Let the set
$\mathcal{C}=\\{c_{1},c_{2},\cdots,c_{n}\\}$, $c_{i}=(x_{i},y_{i})$ indicates
the set of $n$ different points such that each $c_{i}$ indicates a pixel
corresponding to the center of mass of the $i$-th saliency area. To make the
rest of the computation scale independent from the image size, we update the
properties of each point $c_{i}$ as $s_{i}=(x_{i}/I_{x},y_{i}/I_{y})$ such
that $I_{x}$ and $I_{y}$ are the horizontal and vertical size of the image.
Then we build up a complete weighted graph given the set $\mathcal{C}$ such
that the weight $w_{i,j}$ between two vertices $c_{i},c_{j}$ is calculated as
$w_{i,j}=\|s_{i}-s_{j}\|_{2}$. Then we return the summation of all edge
weights as the distance between connected components.

  * •

$f_{40}$: distance from the rule of third points. Professional photographers
usually locate their main object in one of the four interest points based on
the rule of third. The four interest points in rule of third is the
intersection of two vertical and two horizontal lines dividing the image into
$9$ equal segments. Figure 3 shows the four interested points based on rule of
third. This is an important feature in photo beauty evaluation [17],
motivating us to investigate its effect in creative performance. We define
this feature as the minimum distance from the center of mass of the largest
saliency area to one of the four interest points based on rule of third.

Figure 3: The four interested points based on rule of third.

  * •

$f_{41}$: distance from the center of image. This feature is the distance of
saliency components to the center of image which is the most focused part of
an image. The overall distance from the centers of all connected components to
the center of image is returned as feature $f_{41}$. Note that for both
features $f_{40}$ and $f_{41}$, we normalize the position of each pixel
similar as feature $f_{39}$.

####  3.3.2 Number of Characters

We consider the number of characters in an image as feature $f_{42}$. We tried
a number of OCR toolbox and one of them provides us with appropriate results
considering the number of characters in ads[22]. Note that we are interested
in the number of characters in the image regardless of its meaning. To
evaluate the accuracy of the OCR toolbox, we counted the true number of
characters in $100$ random images and compared it to the returned number of
characters from the OCR toolbox. We found strong linear correlation of $0.80$,
suggesting that our toolbox is reasonably accurate in evaluating the number of
characters in images. Note that extracting the exact text from ad creatives is
challenging as they often appear in different fonts, sizes and orientations.

####  3.3.3 Number of Faces

The last feature, $f_{43}$, captures the effect of the human face appearance
on creative performance. In [13] the authors concluded that the human
appearance in an image could make the image more memorable. This motivates us
to test whether face appearance affects creative performance. We count the
number of faces in an image using an available toolbox [16]. Our toolbox is
reasonably accurate and has a correlation more than $0.9$ with the true number
of faces in images in our experiments with a sample size of $100$.

##  4 Experimental Results

In this section we present the algorithms and experiments we designed to
evaluate the relationship between visual features and the performance of
creatives in online display advertising.

###  4.1 Data Set

We extracted creatives of advertising campaigns from the world’s largest
online advertising exchange system, RightMedia. We filtered out animated
creatives because our features are designed for static images. We also
calculated the average CTR of these creatives from online serving history log
during a two-month period.

As discussed in Section 1, the performance of creatives is determined by many
factors. One important factor is the ad position in the webpage. Generally the
available position of a creative on a webpage is determined by the creative’s
size. To remove the impact on performance introduced by ad position (and
size), we create two different data sets, each of which consists of creatives
with the same size. The first data set, ID$2$, consists of $6272$ creatives
with size $250\times 300$ pixels, and the second data set, ID$6$, includes
$3888$ images with $90\times 730$ pixels. All of the creatives have a minimum
of $100K$ impressions guaranteeing that their CTRs have converged to their
true values. The CTR distribution of each data set is shown in Figure 4.

(a) Data set ID2

(b) Data set ID6

Figure 4: The CTR distribution of two data sets.

We further created two sub-categories from data set ID$2$: “dating” with $927$
images and “traveling” with $599$ images. Since there are not many images in
these two categories, we consider the images with a minimum of $20k$ and $10k$
impressions for “dating” and “traveling” respectively.

###  4.2 Learning Methods

The main goal of this work is to study the relationship between the
performance of creatives and their visual features. In the first step we try
to predict CTR from visual features using regression methods. We used three
different regression algorithms to predict CTR, 1) Linear Regression (LR), 2)
Support Vector Regression with RBF kernel(SVR), and 3) Constrained Lasso
(C-Lasso) which is a modification to Lasso [27].

We used LIBSVM [3] to implement the SVR and performed cross validation to
determine the parameters of the model. We describe our constrained Lasso
optimization approach as follows. Suppose we have a set of $n$ creatives at
disposal and the visual features of these creatives are represented as a
matrix $A\in\textbf{R}^{d\times n}$ such that
$A=(\mathbf{a}_{1},\mathbf{a}_{2},\cdots,\mathbf{a}_{n})$ where
$\mathbf{a}_{k}\in\textbf{R}^{d}$ is a column vector representing the $d$
dimensional visual features of creative $k$. In our experiment $d=43$. The CTR
values of the $n$ creatives are represented as a vector
$\mathbf{y}=({y_{1},\cdots,y_{n}})^{\top}\in\textbf{R}^{n}$ where each $y_{k}$
is the CTR of the $k$-th creative. We bound the CTR of each creative by
$y_{min}\leq y_{i}\leq y_{\max}$ where $y_{min}$ and $y_{max}$ can be obtained
from online serving history log. To predict CTR of the creatives, we try to
solve the following optimization problem:

$\displaystyle\min_{w}$
$\displaystyle\|A^{\top}\mathbf{w}-\mathbf{y}\|_{F}^{2}+\lambda\|\mathbf{w}\|_{1}$
(3) $\displaystyle s.t.$ $\displaystyle y_{min}\leq A^{\top}\mathbf{w}\leq
y_{max}$

where $\|\cdot\|_{F}^{2}$ is Frobinius-2 norm and $\|\cdot\|_{1}$ is
$\ell_{1}$ norm, also called lasso. We call the above optimization problem as
constrained Lasso (C-Lasso) and we used [11] to find the solution of this
optimization problem. Note that the proposed C-Lasso approach performs better
than Lasso in our application.

###  4.3 Evaluation

In this section we present different evaluation methods to analyze the
efficacy of the developed visual features in predicting the performance of
creatives.

####  4.3.1 CTR Prediction

|  |  |
---|---|---|---
ID$2$ | ID$6$ | ID$2$-Dating | ID$2$-Traveling, IMP$\geq 10k$
Figure 5: The amount of preserved ranking for each method.

To evaluate the CTR prediction accuracy of the algorithms, we run each
algorithm for $200$ independent runs where in each run $80\%$ of each data set
is selected randomly for training and $20\%$ for testing. The accuracy
evaluation results are reported over the prediction of the test data. Mean
Squared Error (MSE) is used to measure the prediction accuracy for each
algorithm as follows:

$MSE=\frac{1}{n}\sum_{k=1}^{n}|y_{k}-\hat{y}_{k}|^{2}$ (4)

where $n$ is the number of test samples, $y_{k}$ is the true CTR of the $k$-th
creative calculated from history log, and $\hat{y}_{k}$ is the predicted CTR.
To meaningfully interpret the MSE value, we introduce two baseline approaches,
_Random_ and _Constant Mean_(CM) policy.

Table 1: The prediction accuracy of each method against _Random_ policy. Data set | Samples | CM | LR | C-Lasso | SVR
---|---|---|---|---|---
ID$2$ | $6272$ | $1.71$ | $2.28$ | $2.22$ | 3.27
ID$6$ | $3888$ | $1.75$ | $2.27$ | $2.14$ | 2.77
ID$2$-Dating | $927$ | $1.79$ | $2.65$ | $2.58$ | 2.79
ID$2$-Traveling | $599$ | $1.68$ | $2.13$ | $2.03$ | 2.26

The _Random_ policy simply samples from the CTR distribution of the training
data to predict the CTR of each testing creative, while the CM policy assigns
a constant value, $c_{m}$, to all ads where $c_{m}$ is the mean CTR of the
training data. Table 1 shows the average results over $200$ independent runs
for each algorithm. Each entry is the MSE value of the random policy divided
by MSE value of each algorithm. Results show that we can perform up to $3.27$
times better than _Random_ policy in predicting the CTR from visual features
only. All learners perform consistently better than baseline CM as well. This
result demonstrates the non-trivial impact of visual appearance of the
creative on its advertising performance.

####  4.3.2 CTR Ranking

We introduce a ranking criterion to investigate the ability of using visual
features to rank the creatives by their CTRs. Given a test set of creatives,
suppose $c_{1}^{-},c_{2}^{-},\cdots,c_{k}^{-}$ represent the $k$ images with
the lowest CTR values and $c_{1}^{+},c_{2}^{+},\cdots,c_{k}^{+}$ represent the
$k$ images with the highest CTR. Therefore we have $k^{2}$ pairs
$(c_{i}^{-},c_{j}^{+})$ such that $ctr(c_{i}^{-})\leq ctr(c_{j}^{+})$ for
$i,j\in\\{1,\cdots,k\\}$. We wish to know whether our prediction of CTR using
visual features preserves the ranking of pairs $(c_{i}^{-},c_{j}^{+})$. To
test this, we change the value of $k$ as a function of test data size. We then
measure the percentage of match between the predicted ranking of creatives,
and the truly observed ranking in the test data. The results over $200$
independent runs are shown in figure 5 for different data sets. The $x-$axis
indicates the value of $\delta$ such that $k=\delta n$ for
$\delta=0.02,0.04,0.06,\cdots,0.50$ where $n$ is the number of creatives in
the data set, and $y-$axis represents the percentage of correctly ranked
pairs.

Results show that SVR consistently outperforms other learners. As we increase
the size of $k$, the percentage of correctly ranked predictions decreases for
all learning algorithms. This is as expected, since differentiating the images
of creatives which have CTRs close to the mean of the CTR distribution, using
visual features only, is very difficult even for a human. Interestingly, the
results show that by just using visual features, we can preserve more than
$90\%$ of the ranking for data set ID$2$ (for $\delta=0.1$). This number
remains high at $75\%$ when we consider all top-half images against low-half
images for all data sets ($\delta=0.5$). This is an encouraging result that
demonstrates the utility of visual features in predicting the ranking of CTR.

####  4.3.3 CTR Classification

Previous studies in beauty evaluation [7, 15, 17] mostly try to classify the
images into high and low quality category rather than assigning scores to
their beauty based on visual features. Similarly, we evaluate the performance
of classifying the creatives into high (+1) and low (-1) CTR category using
visual features only. We use support vector machine with RBF kernel as our
classifier. Similar to the previous section, we randomly separate $80\%$ of
data as training and use the rest as testing data. Then, we train our
classifier on creatives that belong to the top and bottom $30\%$ in CTR. In
fact, we are disregarding $40\%$ of data that are close to the training data
CTR mean, $\mu_{t}$, to reduce the noise for the classifier. Similar to the
ranking experiments, we filter our test set by focusing on the $k$ creatives
with highest CTR values (labeled as positive) and the $k$ creatives with the
lowest CTR values (labeled as negative), where $k=\delta n$ is varied by
changing $\delta$. We obtain the classification accuracy by comparing the
predicted classes to the true classes obtained from real CTR values. Figure 6
demonstrates the average classification accuracy over $200$ independent runs
where each run uses randomly selected training and testing data. The $x$-axis
indicates the value of $\delta$ and $y$-axis represents the classification
accuracy for each data set given a fixed value of $\delta$. As seen in the
figure, using visual features yields a classification accuracy of $70\%$ when
$\delta=0.5$. Together with the previous results on predicting and ranking
CTR, these results show the efficacy of using visual features of creatives in
predicting CTR.

---
Figure 6: The classification accuracy for each data set.

###  4.4 Feature Selection

The above analysis shows that visual features are useful in predicting the
performance of creatives in online advertising. A natural question is to
identify the visual features that have strong impact on ad performance. Such
information could be very useful in many areas. For example, human graphic
designers may use this information to guide their design of high-performance
creatives. Smart ads system may use this information to dynamically generate
creatives that are more appealing to online users. Ad exchange system may use
this information to determine which creative will win in the auction
marketplace for each advertising opportunity. In this section we conduct a
series of experiments to select such important visual features.

We first calculate the Linear Correlation (LC) and Mutual Information (MI)
between all features and CTR in each data set. Mutual information can provide
us with the information of non-linear correlation between features. Note that,
to calculate the mutual information between any pair of features $(X,Y)$, we
discretized each feature and CTR values into $50$ equal intervals.The results
are shown in table 3. The top $5$ features in each data set with highest
absolute values are highlighted in bold. The table shows that there is no
feature with high linear correlation or mutual information except $f_{12}$ in
data set ID$6$. Thus we use Forward Feature Selection (FFS) to select the top
$k$ features.

Before running FFS, we first cluster the features based on the Normalized
Mutual Information (NMI) of all feature pairs. We discretize each feature into
$50$ equal intervals, and calculate NMI as follows:

$NMI(X;Y)=\frac{I(X;Y)}{\sqrt{H(X)H(Y)}},$ (5)

where $H(X)$ is the entropy of random variable $X$. Then we cluster the
features using the average linkage algorithm [21]. Two clusters are merged
into one if their average NMI is at least $0.2$. This results in $20$ clusters
for data set ID$2$ and $21$ clusters for data set ID$6$. The resulting
clusters are shown in Table 3. In the table, $S_{i}$ represents a set of
features in cluster $i$. We now apply a simple change to the FFS algorithm to
select the top $k$ clusters rather than features. After selecting a feature by
FFS, all the correlated features that belong to the same cluster are removed
from the next steps of FFS. The selected top $k=10$ clusters are shown in
table 2. Note that clustering the features in the above manner helps select
different features (or feature sets) that are less correlated with each other.
For example, all color harmony features are in the same cluster $S_{4}$.
Therefore by selecting one of the features from this cluster, we indicate the
importance of color harmony in CTR, and by removing the highly correlated
features at each step in FFS, we can guarantee to select a set of features
which are less correlated with each other. Below we investigate some of the
selected clusters that are common to both data sets.

Table 2: The top 10 selected clusters by FFS. Data Set | Selected Clusters
---|---
ID$2$ |  $S_{5}$, $S_{1}$, $S_{19}$, $S_{17}$, $S_{13}$, $S_{18}$, $S_{20}$, $S_{10}$, $S_{11}$, $S_{9}$
ID$6$ |  $S_{1}$, $S_{2}$, $S_{20}$, $S_{5}$, $S_{17}$, $S_{14}$, $S_{9}$, $S_{13}$, $S_{4}$, $S_{18}$

Table 2 shows that $S_{1}$ is the best feature set (or cluster) for data set
ID$6$ and the second best set for data set ID$2$ which illustrates the
importance of set $S_{1}$. $S_{1}$ consists of the gray level features $f_{1}$
and $f_{2}$ of the image. The scatter plot of both features in data set ID$2$
is shown in Figure7 (the scatter plot in data set ID$6$ is similar). Figure 7
shows that for creatives with small value in both features, high CTR value is
unlikely, and creatives with high CTR values should have high values in these
two features. This is consistent with the intuition that creatives with higher
contrast should perform better. Note that having high values in these two
features does not guarantee a high CTR value.

|
---|---
Figure 7: The scatter plot of $f_{1}$ and $f_{2}$ against CTR.

$S_{5}$ is the best feature set for data set ID$2$ and the fourth for ID6. It
only includes $f_{10}$ which is the number of connected coherent components.
The scatter plot of $f_{10}$ in both data sets are shown in figure 8. The
scatter plot shows that creatives with more than $15$ connected coherent
components in data set ID$6$ and more than $20$ in data set ID$2$ are unlikely
to achieve a CTR higher than $0.01$. In other words, this suggests that
cluttered creatives containing many objects tend to have lower CTR.

|
---|---
Figure 8: The scatter plot of $f_{10}$.

The number of characters, $S_{19}$ in data set ID$2$ and $S_{20}$ in data set
ID$6$, is interestingly the third important feature set in both data sets.
Figure 9 shows the scatter plot of the number of characters in both data sets.
It can be seen that the creatives with higher number of characters are
unlikely to achieve high CTR values in both data sets, once again suggesting
that textual clutter is undesirable.

|
---|---
ID$2$ | ID$6$
Figure 9: The scatter plot of number of characters.

The next selected categories is $S_{17}$ which is the $4$-th selected category
in ID$2$ and the $5$-th in ID$6$. $S_{17}$ represents the number of connected
components in saliency binary map, distance between salient components,
distance of saliency areas from the center of image and rule of third closest
point. This indicates the importance of saliency features as well as
considering professional photography rules such as the rule of third in
designing ads. Intuitively, a small number of salient components, closer to
the center of the creative, and consistent with the rule of third are
desirable features in a creative. Finally, $S_{13}$, which contains features
describing the number of hues and the contrast of hues in the largest segment
of the image, is the $5$-th important common category considering both data
sets. Note that the scatter plot of the last $2$ selected categories have been
omitted due to space limit. In summary, our top $5$ selected categories
include the features from all proposed feature categories, global, local and
advanced features, indicating the importance of each of them in predicting the
creatives CTR.

Table 3: Complete list of visual features. Feature | Short Description | LC ID$2$ | MI ID$2$ | LC ID$6$ | MI ID$6$ | C-ID$2$ | C-ID$6$
---|---|---|---|---|---|---|---
$f_{1}$ | gray level contrast | -0.048 | 0.063 | -0.102 | 0.098 | $S_{1}$ | $S_{1}$
$f_{2}$ | number of dominant bins in gray level histogram | -0.107 | $0.071$ | -0.137 | 0.112 | $S_{1}$ | $S_{1}$
$f_{3}$ | standard deviation of gray level images | -0.063 | 0.061 | -0.016 | 0.082 | $S_{2}$ | $S_{2}$
$f_{4}$ | number of dominant bins in RGB histogram | -0.083 | 0.067 | -0.074 | 0.078 | $S_{3}$ | $S_{3}$
$f_{5}$ | size of the dominant bin in RGB histogram | 0.078 | 0.093 | 0.064 | 0.104 | $S_{3}$ | $S_{3}$
$f_{6}$ | number of dominant bins in HSV histogram | -0.083 | 0.073 | -0.078 | 0.092 | $S_{3}$ | $S_{3}$
$f_{7}$ | size of the dominant bin in HSV histogram | 0.119 | 0.096 | 0.085 | 0.094 | $S_{3}$ | $S_{3}$
$f_{8}$ | deviation from the best color harmony model | -0.062 | 0.039 | -0.040 | 0.052 | $S_{4}$ | $S_{4}$
$f_{9}$ | average deviation from the best two color harmony models | -0.069 | 0.047 | -0.046 | 0.063 | $S_{4}$ | $S_{4}$
$f_{10}$ | number of connected coherent components | 0.094 | 0.062 | -0.004 | 0.064 | $S_{5}$ | $S_{5}$
$f_{11}$ | size of the largest connected coherent component | 0.102 | 0.090 | 0.069 | 0.099 | $S_{3}$ | $S_{3}$
$f_{12}$ | size of the second largest connected coherent component | 0.085 | 0.068 | 0.315 | 0.122 | $S_{6}$ | $S_{6}$
$f_{13}$ | color size rank of the largest connected coherent component | -0.002 | 0.010 | -0.012 | 0.014 | $S_{7}$ | $S_{7}$
$f_{14}$ | color size rank of the second largest connected coherent component | -0.046 | 0.020 | -0.039 | 0.022 | $S_{8}$ | $S_{8}$
$f_{15}$ | number of dominant hues | -0.102 | 0.036 | -0.119 | 0.045 | $S_{9}$ | $S_{9}$
$f_{16}$ | contrast of dominant hues | -0.015 | 0.027 | -0.015 | 0.046 | $S_{9}$ | $S_{9}$
$f_{17}$ | standard deviation of hues | 0.010 | 0.084 | -0.039 | 0.102 | $S_{10}$ | $S_{10}$
$f_{18}$ | average lightness | 0.031 | 0.072 | 0.035 | 0.099 | $S_{11}$ | $S_{11}$
$f_{19}$ | standard deviation of lightness | -0.065 | 0.063 | -0.086 | 0.084 | $S_{2}$ | $S_{2}$
$f_{20}$ | size of the Largest Segments (LS) | -0.052 | 0.058 | -0.080 | 0.081 | $S_{12}$ | $S_{12}$
$f_{21}$ | segments size contrast | -0.044 | 0.054 | -0.122 | 0.092 | $S_{12}$ | $S_{12}$
$f_{22}$ | number of image dominant hues in the LS | -0.104 | 0.044 | -0.068 | 0.029 | $S_{13}$ | $S_{13}$
$f_{23}$ | number of dominant hues in the LS | -0.102 | 0.032 | -0.080 | 0.036 | $S_{13}$ | $S_{13}$
$f_{24}$ | largest number of dominant hues in one segment | -0.079 | 0.031 | -0.125 | 0.045 | $S_{14}$ | $S_{14}$
$f_{25}$ | contrast of hues number among segments | -0.006 | 0.027 | -0.098 | 0.033 | $S_{14}$ | $S_{14}$
$f_{26}$ | contrast of hues in the LS | -0.120 | 0.032 | -0.085 | 0.036 | $S_{13}$ | $S_{13}$
$f_{27}$ | standard deviation of hues contrast among segments | -0.056 | 0.072 | -0.073 | 0.098 | $S_{10}$ | $S_{10}$
$f_{28}$ | deviation from the best color harmony model for LS | -0.047 | 0.030 | -0.034 | 0.042 | $S_{4}$ | $S_{4}$
$f_{29}$ | average deviation from the best two color harmony models for LS | -0.055 | 0.037 | -0.049 | 0.053 | $S_{4}$ | $S_{4}$
$f_{30}$ | average lightness in LS | 0.027 | 0.075 | 0.065 | 0.096 | $S_{11}$ | $S_{11}$
$f_{31}$ | standard deviation of average lightness among the segments | 0.080 | 0.076 | -0.049 | 0.091 | $S_{15}$ | $S_{15}$
$f_{32}$ | contrast of average lightness among the segments | 0.075 | 0.067 | -0.027 | 0.081 | $S_{15}$ | $S_{15}$
$f_{33}$ | background size in Saliency Map (SM) | -0.165 | 0.082 | -0.175 | 0.106 | $S_{16}$ | $S_{16}$
$f_{34}$ | number of connected components in SM | -0.111 | 0.029 | -0.062 | 0.088 | $S_{17}$ | $S_{17}$
$f_{35}$ | size of the largest connected components in SM | 0.132 | 0.075 | 0.036 | 0.073 | $S_{16}$ | $S_{18}$
$f_{36}$ | average saliency weight of largest connected component | 0.165 | 0.075 | 0.071 | 0.080 | $S_{16}$ | $S_{18}$
$f_{37}$ | number of connected components in image background SM | -0.012 | 0.015 | -0.035 | 0.038 | $S_{18}$ | $S_{19}$
$f_{38}$ | size of the largest connected component of background in SM | -0.163 | 0.091 | -0.145 | 0.052 | $S_{16}$ | $S_{16}$
$f_{39}$ | distance between connected components in SM | 0.117 | 0.056 | -0.027 | 0.072 | $S_{17}$ | $S_{17}$
$f_{40}$ | distance from rule of third points in SM | 0.126 | 0.063 | 0.010 | 0.086 | $S_{17}$ | $S_{17}$
$f_{41}$ | distance from center of image in SM | 0.091 | 0.065 | 0.011 | 0.087 | $S_{17}$ | $S_{17}$
$f_{42}$ | number of characters (OCR) | -0.082 | 0.057 | -0.017 | 0.071 | $S_{19}$ | $S_{20}$
$f_{43}$ | number of faces in the image | -0.035 | 0.008 | -0.024 | 0.011 | $S_{20}$ | $S_{21}$

##  5 Conclusion

In this paper we investigated the relationship between the user response rate
and the visual appearance of creatives in online display advertising. To the
best of our knowledge, this is the first work in this area. We designed $43$
visual features for our experiments. We extracted the features from large
scale data produced by the world’s largest ad exchange system. We tested the
utility of visual features in CTR prediction, ranking and classification. The
experimental results demonstrate that our proposed framework is able to
outperform baseline consistently, indicating the efficacy of visual features
in predicting CTR. We also performed feature selection to select the top
visual feature categories that have strongest importance for increasing CTR.
The findings from this work will be useful for ads selection and developing
visually appealing creatives with higher user response propensity in online
display advertising.

## References

  * Bovik et al. [2005] Bovik, A. C., Cormack, L. K., Aggarwal, J. K., Bajaj, C., Veciana, G. D., Ghosh, J., Sheikh, H. R., Sheikh, H. R., and Sc, B. (2005).  No-reference quality assessment using natural scene statistics: Jpeg2000.  JPEG2000.
  * Chakrabarti et al. [2008] Chakrabarti, D., Agarwal, D., and Josifovski, V. (2008).  Contextual advertising by combining relevance with click feedback.  In WWW, pages 417–426.
  * Chang and Lin [2011] Chang, C.-C. and Lin, C.-J. (2011).  LIBSVM: A library for support vector machines.  ACM Transactions on Intelligent Systems and Technology.
  * Chatterjee [2008] Chatterjee, P. (2008).  Are unclicked ads wasted? enduring effects of banner and pop-up ad exposures on brand memory and attitudes.  Journal of Electronic Commerce Research, pages 51–61.
  * Cohen-Or et al. [2006] Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., and Xu, Y.-Q. (2006).  Color harmonization.  In ACM Transactions on Graphics, pages 624–630.
  * Damera-venkata et al. [2000] Damera-venkata, N., Member, S., Kite, T. D., Geisler, W. S., Evans, B. L., Member, S., and Bovik, A. C. (2000).  Image quality assessment based on a degradation model.  IEEE Trans. Image Processing, pages 636–650.
  * Datta et al. [2006] Datta, R., Joshi, D., Li, J., and Wang, J. Z. (2006).  Studying aesthetics in photographic images using a computational approach.  In ECCV, pages 7–13.
  * Donthu et al. [2004] Donthu, N., Lohtia, R., Osmonbekov, T., and Xie, T. (2004).  Emotional appeal and incentive offering in banner advertisements.  Journal of Interactive Advertising, pages 30–37.
  * Goldfarb and Tucker [2010] Goldfarb, A. and Tucker, C. (2010).  Online display advertising: Targeting and obtrusiveness.  Marketing Science.
  * Gooch et al. [2001] Gooch, B., Reinhard, E., Moulding, C., and Shirley, P. (2001).  Artistic composition for image creation.  In Proceedings of the 12th Eurographics Workshop on Rendering Techniques, pages 83–88.
  * Grant and Boyd [2011] Grant, M. and Boyd, S. (2011).  CVX: Matlab software for disciplined convex programming, version 1.21.
  * Hou and Zhang [2007] Hou, X. and Zhang, L. (2007).  Saliency detection: A spectral residual approach.  IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1–8.
  * Isola et al. [2011] Isola, P., Xiao, J., Torralba, A., and Oliva, A. (2011).  What makes an image memorable?  In IEEE Conference on Computer Vision and Pattern Recognition, pages 145–152.
  * Itti et al. [1998] Itti, L., Koch, C., and Niebur, E. (1998).  A model of saliency-based visual attention for rapid scene analysis.  IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1254–1259.
  * Ke et al. [2006] Ke, Y., Tang, X., and Jing, F. (2006).  The design of high-level features for photo quality assessment.  In CVPR, pages 419–426.
  * Krishna [2008] Krishna, S. (2008).  Open cv viola-jones face detection.
  * Li and Chen [2009] Li, C. and Chen, T. (2009).  Aesthetic visual quality assessment of paintings.  IEEE Journal of Selected Topics in Signal Processing, pages 236–252.
  * Li and Bukovac [1999] Li, H. and Bukovac, J. L. (1999).  Cognitive impact of banner ad characteristics: An experimental study.  Journalism and Mass Communication Quarterly, pages 341–53.
  * Luo and Tang [2008] Luo, Y. and Tang, X. (2008).  Photo and video quality evaluation: Focusing on the subject.  In ECCV, pages 386–399.
  * McQuarrie and Mick [1999] McQuarrie, E. and Mick, D. (1999).  Visual rhetoric in advertising: text-interpretive, experimental, and reader-response analyses.  Journal of consumer research, pages 37–54.
  * Murtagh [1983] Murtagh, F. (1983).  A survey of recent advances in hierarchical clustering algorithms.  The Computer Journal, pages 354–359.
  * Orlando [2007] Orlando, D. (2007).  Optical character recognition.
  * Pass et al. [1996] Pass, G., Zabih, R., and Miller, J. (1996).  Comparing images using color coherence vectors.  In Fourth ACM international conference on Multimedia, pages 65–73.
  * Renjie et al. [2010] Renjie, L. L., Lior, C., Cohen-or, W. D., Liu, L., Chen, R., Wolf, L., and Cohen-or, D. (2010).  Optimizing photo composition.  In Computer Graphics Forum, pages 469–478.
  * Richardson [2007] Richardson, M. (2007).  Predicting clicks: Estimating the click-through rate for new ads.  In WWW, pages 521–530.
  * Shi and Malik [2000] Shi, J. and Malik, J. (2000).  Normalized cuts and image segmentation.  IEEE Trans. Pattern Analysis and Machine Intelligence, pages 888–905.
  * Tibshirani [1996] Tibshirani, R. (1996).  Regression shrinkage and selection via the lasso.  Journal of the Royal Statistical Society, pages 267–288.
  * Tokumaru et al. [2002] Tokumaru, M., Muranaka, N., and Imanishi, S. (2002).  Color design support system considering color harmony.  In Fuzzy Systems, 2002, pages 378–383.
  * Tong et al. [2004] Tong, H., Li, M., jiang Zhang, H., He, J., and Zhang, C. (2004).  Classification of digital photos taken by photographers or home users.  In Proceedings of Pacific Rim Conference on Multimedia, pages 198–205.
  * Yaveroglu and Donthu [2008] Yaveroglu, I. and Donthu, N. (2008).  Advertising repetition and placement issues in on-line environments.  Journal of Advertising, pages 31–44.
  * Y.Mastuda [1995] Y.Mastuda (1995).  Color design.  In Asakura Shoten.