[1,2,4]Huck Stepanyants,

[1,2,4,5]Dmitri Krioukov

1]Department of Physics, Northeastern University, Boston, 02115,
Massachusetts, USA

2]Network Science Institute, Northeastern University, Boston, 02115,
Massachusetts, USA

3]Department of Pure Mathematics and Mathematical Statistics, University of
Cambridge, Cambridge, CB3 0WB, UK

4]Department of Mathematics, Northeastern University, Boston, 02115,
Massachusetts, USA

5]Department of Electrical and Computer Engineering, Northeastern University,
Boston, 02115, Massachusetts, USA

# Diameter of compact Riemann surfaces

<EMAIL_ADDRESS>Alan Beardon<EMAIL_ADDRESS>Jeremy Paton
<EMAIL_ADDRESS><EMAIL_ADDRESS>[  [  [  [  [

###### Abstract

Diameter is one of the most basic properties of a geometric object, while
Riemann surfaces are one of the most basic geometric objects. Surprisingly,
the diameter of compact Riemann surfaces is known exactly only for the sphere
and the torus. For higher genuses, only very general but loose upper and lower
bounds are available. The problem of calculating the diameter exactly has been
intractable since there is no simple expression for the distance between a
pair of points on a high-genus surface. Here we prove that the diameters of a
class of simple Riemann surfaces known as _generalized Bolza surfaces_ of any
genus greater than $1$ are equal to the radii of their fundamental polygons.
This appears to be the first exact result for the diameter of a compact
hyperbolic manifold.

###### keywords:

Diameter, Riemann surfaces, hyperbolic manifolds

##  1 Introduction

A Riemann surface is any connected, one-dimensional complex manifold.
According to the classification theorem of closed surfaces Seifert , any
compact Riemann surface is homeomorphic to either the sphere or the connected
sum of $g$ tori, where $g$ is the genus of the surface. Once equipped with a
metric, a Riemann surface becomes a Riemannian manifold. The sphere $(g=0)$
admits the spherical metric, the torus $(g=1)$ admits the Euclidean metric,
while the surfaces of genus $g>1$ admit the hyperbolic metric.

The diameter $\mathscr{D}$ of a metric space is the maximum distance between a
pair of points in it. Diameter is one of the most basic characteristics of any
geometric object. Surprisingly, the diameter of compact Riemann surfaces is
known exactly only for the sphere and the torus. The best results on the
diameter of surfaces of genus $g>1$ are only loose lower and upper bounds in
terms of the total surface area, the systole, and the genus Chang ; Balacheff
; Budzinski ; Bavard . The _systole_ is the length of a shortest
noncontractible loop on the surface.

Our main result is a proof of the following theorem:

######  Theorem 1.

The diameter of a class of Riemann surfaces $S_{g}$, known as generalized
Bolza surfaces, of genus $g>1$ is
$\mathscr{D}_{g}=\textnormal{arccosh}\left(\cot^{2}(\pi/4g)\right)$.

The definition of the surfaces $S_{g}$ is in the next section. In particular,
$S_{2}$ is known as the Bolza surface Bolza , one of the first compact
hyperbolic manifolds ever considered. A free particle moving along a geodesic
on the Bolza surface was the first dynamical system proven rigorously to be
chaotic Hadamard . The Bolza surface is also known to maximize the systole
across all genus-$2$ surfaces Werner . For ${g>2}$, the $S_{g}$ are the
generalized Bolza surfaces Ebbens . They appear frequently in studies of
hyperbolic surfaces due to their high degree of symmetry Wiman ; Ebbens ;
Bujalance1 ; Bujalance2 .

Knowing the diameter of a surface, we can efficiently compute the distance
between any pair of points on it, a result to be published elsewhere. Theorem
1 states that the Bolza surface has diameter
$\mathscr{D}=\textnormal{arccosh}\left(3+2\sqrt{2}\right)\approx 2.45$. To the
best of our knowledge, this is the first ever exact result for the diameter of
_a_ compact hyperbolic manifold. The closest results to ours appear to be the
ones in Nabutovsky2003 . They apply to manifolds of dimension at least five.

We proceed by collecting all the necessary background information and
definitions in Section 2. Section 3 contains the outline of the proof of
Theorem 1 split into a sequence of theorems that we state in that section as
well. We prove all those theorems in the concluding Section 4.

##  2 Background information and definitions

We use the Poincaré disk model of the hyperbolic plane $\mathbb{H}^{2}$. The
isometries (distance-preserving maps) of $\mathbb{H}^{2}$ are given by the
matrices

$\left[\begin{matrix}a&\overline{c}\\\ c&\overline{a}\end{matrix}\right],\quad
a,c\in\mathbb{C},~{}\lvert a\rvert^{2}-\lvert c\rvert^{2}=1,$ (1)

which form a subgroup of $PSL(2,\mathbb{C})$, the invertible ${2\times 2}$
complex matrices. The action of each matrix on a complex number $z$ in the
Poincaré disk ${\mathbb{D}=\\{z\in\mathbb{C}:\lvert z\rvert<1\\}}$ is a
fractional linear transformation:

$\left[\begin{matrix}a&\overline{c}\\\
c&\overline{a}\end{matrix}\right](z)=\frac{az+\overline{c}}{cz+\overline{a}}.$
(2)

A _Fuchsian group_ $\mathscr{F}$ is a discrete subgroup of
${PSL(2,\mathbb{C})}$ that has an invariant disk in ${\mathbb{C}^{\infty}}$.
Each $\mathscr{F}$ defines a hyperbolic Riemann surface $S$ which is the
quotient surface ${S=\mathbb{H}^{2}/\mathscr{F}}$. A _fundamental domain_ is
an open, connected set in ${\mathbb{H}^{2}}$ that contains at most one
representative of each point on ${S}$, and whose closure contains at least one
representative of each point on ${S}$ Beardon . A fundamental domain and its
images under the actions of $\mathscr{F}$ tessellate $\mathbb{H}^{2}$.

Since points on a quotient surface $S$ are cosets in
$\mathbb{H}^{2}/\mathscr{F}$, the distance between two points (cosets)
$[z]=\mathscr{F}z,[w]=\mathscr{F}w$ on $S$ is given by

$\delta^{\star}([z],[w])=\inf\\{\delta(z^{\prime},w^{\prime}),z^{\prime}\in[z],w^{\prime}\in[w]\\},$
(3)

where $\delta(z,w)$ denotes the distance in $\mathbb{H}^{2}$. The diameter
$\mathscr{D}$ of $S$ is the largest distance between two points on $S$.

The best results on the diameters of Riemann surfaces of genus $g>1$ are as
follows. It was shown in Chang  that for any such surface, the following
inequalities hold, where $\ell$ is the systole, $A$ is the total area of the
surface, and $\mathscr{D}$ is the diameter:

$\displaystyle 2\ell\sinh(\mathscr{D})$ $\displaystyle\geq A\text{, and }$ (4)
$\displaystyle 2\sinh(\ell/4)\mathscr{D}$ $\displaystyle\leq A.$ (5)

In Balacheff  it was shown that

$4\cosh(\ell/2)\leq 3\cosh(\mathscr{D})-1.$ (6)

Another lower bound exists in terms of the area alone Budzinski :

$\cosh\mathscr{D}\geq\frac{A}{2\pi}+1,$ (7)

and finally, there is the following lower bound in terms of the genus Bavard :

$\cosh\mathscr{D}\geq\frac{1}{\sqrt{3}}\cot\left(\frac{\pi}{6(2g-1)}\right).$
(8)

All of the bounds above hold in general for any Riemann surface, as do other
related spectral results Cheng1 ; Cheng2 ; Cheeger . However, none of them is
tight for the ${S_{g}}$.

Figure 1: Left: The Bolza surface $(S_{2})$ and its “gluing” scheme. Sides
that are identified in the quotient space are labeled with the same color, and
the arrows represent the actions of the generators ${\\{t_{k}\\}}$ and their
inverses on the fundamental polygon. Right: $S_{3}$ and its gluing scheme.

The _generalized Bolza surface_ ${S_{g}}$ of genus ${g}$ is defined Ebbens ;
Aurich  to be the surface obtained by identifying (“gluing”) the opposite
sides of the regular $4g$-gon with interior angles $\pi/2g$, Fig. 1, whose
side length $s$ and radius $R$ satisfy Aurich

$\cosh^{2}\left(\frac{s}{2}\right)=\cosh
R=\cot^{2}\left(\frac{\pi}{4g}\right),$ (9)

and whose vertices $\\{v_{k}\\}$ are evenly spaced at distance $R$ from the
origin:

$v_{k}=\tanh(R/2)~{}e^{\left(k-\frac{1}{2}\right)\frac{\pi i}{2g}},\quad
k=0,1,\dots,4g-1.$ (10)

This surface $S_{g}$ is the quotient surface
${\mathbb{H}^{2}/\mathscr{F}_{g}}$, where ${\mathscr{F}_{g}}$ is the Fuchsian
group generated by the following ${2g}$ generators and their inverses:

$t_{k}=\left[\begin{matrix}1&\tanh\left(\frac{s}{2}\right)e^{\frac{k\pi
i}{2g}}\\\ \tanh\left(\frac{s}{2}\right)e^{-\frac{k\pi
i}{2g}}&1\end{matrix}\right],\quad k=0,1,\dots,2g-1.$ (11)

These generators glue the opposite sides of the polygon by mapping it to its
edge-adjacent polygons in the tessellation as shown in Fig. 1. By Poincaré’s
theorem Beardon , the interior of the polygon is a fundamental domain of
${S_{g}}$, while the polygon itself is called the _fundamental polygon_.

The surface $S_{2}$ of genus $2$ is the well-known Bolza surface Bolza . The
surfaces ${S_{g}}$ are also known as the _Wiman surfaces of type II_
Bujalance1 ; Bujalance2 . They have ${8g}$ automorphisms, except the ${g=2}$
Bolza surface, which has ${48}$ automorphisms. The ${S_{g}}$ also have
automorphisms of order ${4g}$, which is the second largest possible order, as
was proven in Wiman .

Our main result is the exact diameter of the surface ${S_{g}}$ of genus
${g>1}$, Theorem 1, which in view of Eq. (9) can be restated as:

######  Theorem 1.

${\mathscr{D}_{g}=R}$.

The problem of finding the diameter is not trivial thanks to the definition of
the distance in Eq. (3), which is an infimum over the infinitely many elements
in ${\mathscr{F}_{g}}$. Even though this infimum is actually a minimum since
only a finite subset of group elements needs to be considered, there is no
explicit expression for such a subset, which is the main difficulty in the
problem. In fact, as mentioned in the introduction, our main motivation for
this paper is that the knowledge of the diameter of a surface allows us to
provide an explicit expression for this subset, leading to an efficient
formula to compute distances between pairs of points on the surface, a result
to appear in a follow-on paper.

Our proof of the main result, which we outline in the following section, is a
combination of geometric and algebraic techniques. Specifically, we use some
geometric symmetries of $S_{g}$ to simplify the problem, and algebra to
compute the optimal distances.

##  3 Proof strategy

We first observe that every vertex of the fundamental polygon represents the
same point in the quotient space; we will call this point the _quotient
vertex_ , $[v]\in S_{g}$. Similarly, we define $[0]$ as the point in the
quotient space to which the origin maps. For an arbitrary point $z$ in the
Poincaré disk, let $d_{0}(z)$ and $d_{v}(z)$ denote the quotient distances
from $[z]$ to $[0]$ and $[v]$, respectively.

The proof of Theorem 1 will be broken into several smaller, sequential
theorems. First, we will show that ${\delta^{\star}([0],[v])=R}$, thus proving
the following theorem:

######  Theorem 2.

The diameter $\mathscr{D}_{g}$ of $S_{g}$ is at least $R$,
$\mathscr{D}_{g}\geq R$.

The remaining theorems are aimed at proving $\mathscr{D}_{g}\leq R$, or
equivalently, ${\delta^{\star}([z],[w])\leq R}$ for every pair of points
${[z],[w]}$ in the quotient space. The symmetry of $S_{g}$ allows us to make
several simplifying assumptions about $[z]$ and $[w]$ without loss of
generality.

The symmetries of the quotient surface $S_{g}$ are isometric bijections that
map ${S_{g}}$ to itself. In the Poincaré disk ${\mathbb{D}}$, these symmetries
can be thought of as the subgroup of isometries $\phi$ of ${\mathbb{D}}$ for
which the projection ${\phi^{\star}}$ of ${\phi}$ onto the quotient surface,
given by

$\phi^{\star}([p])=[\phi(p)],$ (12)

is well-defined and isometric. These properties are satisfied by all ${\phi}$
for which the Fuchsian group ${\mathscr{F}_{g}}$, a subgroup of
${PSL(2,\mathbb{C})}$, is invariant under conjugation by ${\phi}$:
${\phi\mathscr{F}_{g}\phi^{-1}=\mathscr{F}_{g}}$. Indeed, for such ${\phi}$
and any ${f,f_{1},f_{2}\in\mathscr{F}_{g}}$ and
${p,p_{1},p_{2}\in\mathbb{D}}$, there exist
${f^{\prime},f_{1}^{\prime},f_{2}^{\prime}\in\mathscr{F}_{g}}$ such that

$\displaystyle\phi^{\star}([fp])=[\phi
f(p)]=[f^{\prime}\phi(p)]=[\phi(p)]=\phi^{\star}([p]),\text{ and}$ (13)
$\displaystyle\delta^{\star}(\phi^{\star}([p_{1}]),\phi^{\star}([p_{2}]))=\delta^{\star}([\phi(p_{1})],[\phi(p_{2})])=$
$\displaystyle\min\\{\delta(f_{1}\phi(p_{1}),f_{2}\phi(p_{2})),f_{1},f_{2}\in\mathscr{F}_{g}\\}=$
$\displaystyle\min\\{\delta(\phi f_{1}^{\prime}(p_{1}),\phi
f_{2}^{\prime}(p_{2})),f_{1}^{\prime},f_{2}^{\prime}\in\mathscr{F}_{g}\\}=$
$\displaystyle\min\\{\delta(f_{1}^{\prime}(p_{1})),\delta(f_{2}^{\prime}(p_{2})),f_{1}^{\prime},f_{2}^{\prime}\in\mathscr{F}_{g}\\}=$
$\displaystyle\delta^{\star}([p_{1}],[p_{2}]),$ (14)

so ${\phi}$ is well-defined and an isometry.

The simplest such isometries are the elements of $\mathscr{F}_{g}$. Indeed,
for ${\phi\in\mathscr{F}_{g}}$ we have
${\phi\mathscr{F}_{g}\phi^{-1}=\mathscr{F}_{g}}$. In this case,
${\phi^{\star}}$ is simply the identity function on ${S_{g}}$. Therefore,
$S_{g}$ is symmetric under actions by elements of ${\mathscr{F}_{g}}$.

Another subgroup of isometries of $\mathbb{D}$ that are symmetries of $S_{g}$
are rotations of $\mathbb{D}$ by ${\pi/4g}$. Although this fact is evident
from the definition of $S_{g}$ in Section 2, we give its formal proof in
Section 4.

Figure 2: The dual tessellation (blue) overlayed against the original
tessellation (black) in the case $g=2$.

The final and most complicated subgroup of symmetries of $S_{g}$ that we will
need is its _dual symmetries_. We define them with the aid of new fundamental
polygons that we call the _dual polygons_ , and the corresponding _dual
tessellation_ of $\mathbb{H}^{2}$. Dual polygons are formed by joining by
geodesics the centers of the $4g$ polygons around the vertices in the original
tessellation, Fig. 2. We will prove the following proposition:

######  Proposition 1.

Let ${\phi}$ be any isometry of $\mathbb{D}$ that maps the fundamental polygon
to any of its dual polygons. Then the projection
$\phi^{\star}:S_{g}\rightarrow S_{g}$ given by

$\phi^{\star}([z])=[\phi(z)]$ (15)

is well-defined and is an isometric automorphism of $S_{g}$. In particular,
dual polygons are fundamental polygons.

The rotational and dual symmetries of $S_{g}$ allow us to reduce drastically
the set of all the possibilities of where the pairs of points $z$ and $w$ can
lie to the following subset:

######  Proposition 2.

To prove Theorem 1, it suffices to consider only the pairs of points $z$ and
$w$ satisfying the following conditions:

  1. 1.

$z$ lies in triangle ${T=\Delta 0v_{1}v_{2}}$, Fig. 3,

  2. 2.

$d_{0}(z)\geq d_{v}(w)$,

  3. 3.

$d_{v}(w)\leq s/2$.

Assuming, thanks to Prop. 2, that the first point $z$ lies in the triangle $T$
with vertices $O,v_{1},v_{2}$, let ${w_{1},w_{2}\in[w]}$ be the two dual-
tessellation representatives of $[w]$ lying in the two dual polygons centered
at $v_{1},v_{2}$, respectively, Fig. 3. To prove our desired result
${\delta^{\star}([z],[w])\leq R}$, it suffices to show that

$\min\left[\delta(z,w_{1}),\delta(z,w_{2})\right]\leq R,$ (16)

simply because

$\displaystyle\delta^{\star}([z],[w])=$
$\displaystyle\min\\{\delta(z^{\prime},w^{\prime}),z^{\prime}\in[z],w^{\prime}\in[w]\\}\leq$
$\displaystyle\min\left[\delta(z,w_{1}),\delta(z,w_{2})\right].$ (17)

Figure 3: Left: The representatives $w_{1}$ and $w_{2}$ of $[w]$ in the dual
tessellation in the ${g=2}$ case. They are images of ${w}$ in the dual
polygons centered at ${v_{1}}$ and ${v_{2}}$. Right: The domain $\Omega_{w}$
is the set of points ${z}$ satisfying the conditions in Prop. 2 for a given
point ${w}$.

Before we prove Eq. (16) using algebra, we take one final step to reduce the
number of possibilities in locations of $z,w$ to consider. Given $w$ such that
$d_{v}(w)\leq s/2$, we define the domain $\Omega_{w}$ as the set of all $z\in
T$ for which $z$ and $w$ meet the conditions listed in Prop. 2, Fig. 3. We
will prove the following:

######  Theorem 3.

Given $w$ for which $d_{v}(w)\leq s/2$, the function

$f(z)=\min\left[\delta(z,w_{1}),\delta(z,w_{2})\right]$

attains its global maximum over the domain $\Omega_{w}$ on its boundary
${\partial\Omega_{w}}$.

This theorem allows us to reduce the possibilities for the location of $z$
even further, to include only the points on the boundary of $\Omega_{w}$.

The final step in the proof of Theorem 1 is to show that Eq. (16) holds for
all of the $z,w$ pairs satisfying all the conditions above:

######  Theorem 4.

Given $w$ for which $d_{v}(w)\leq s/2$, for ${\forall z\in\partial\Omega_{w}}$
we have

$\min\left[\delta(z,w_{1}),\delta(z,w_{2})\right]\leq R.$

Thus we will have shown that $\mathscr{D}_{g}\leq R$, and the combination of
this result with that of Theorem 2 proves Theorem 1.

One important question is how generalizable our proof strategy outlined in
this section is to other surfaces, such as those appearing in the Poincaré
theorem Katok  or the canonical surfaces Abrams . In these surfaces, the
gluing identifies not the opposite sides of a polygon, but its alternating
sides: 1 to 3, 2 to 4, 5 to 7, 6 to 8, and so on. One could hope that if the
polygon is still a regular ${4g}$-gon, then this gluing would lead to a
surface with the same diameter ${R}$. However, this is not true—the diameter
of these surfaces appears to be greater than ${R}$ in simulations. This
observation is a reflection of the fact that the symmetries of a surface play
a crucial role in defining its geometric properties including the diameter, so
one should select very carefully a right set of symmetries of a surface in
calculating its diameter. Our proof is an example of this strategy applied to
a particular class of highly symmetric surfaces. Its generalization to other
surfaces with other groups of symmetry is an interesting open problem.

The next section contains the complete proofs of all the theorems and
propositions above.

##  4 Proofs

######  Theorem 2.

The diameter $\mathscr{D}_{g}$ of $S_{g}$ is at least $R$,
$\mathscr{D}_{g}\geq R$.

Proof: Take ${[z]=[0]}$ and $[w]=[v]$. We will show that
$\delta^{\star}([0],[v])=R$, from which the theorem follows immediately. For
this, we use that the fundamental polygon $P$ is also the Dirichlet polygon of
$[0]$,

$D_{0}=\\{p\mid\delta(0,p)<\delta(f(0),p)\text{ for }\forall
f\in\mathscr{F}_{g},~{}f\neq I\\}.$ (18)

where ${I}$ is the identity. To see this, first note that points $p$ on the
geodesic segment ${\overline{v_{i}v_{i+1}}}$ satisfy
${\delta(0,p)=\delta(t_{i}(0),p)}$ owing to the symmetry of the fundamental
polygon. Next, denote by ${H_{i}}$ the half-plane bounded by this line and
containing $0$. Then

$\displaystyle D_{0}=\\{p\mid\delta(0,p)<\delta(f(0),p)\text{ for }\forall
f\in\mathscr{F}_{g},~{}f\neq I\\}\subseteq$
$\displaystyle\\{\delta(0,p)<\delta(t_{i}(0),p),~{}0\leq i<4g\\}=\cap_{0\leq
i<4g}H_{i},$ (19)

and the last expression is just $P$, thus ${D_{0}\subseteq P}$. Since
Dirichlet polygons are fundamental polygons, which all have the same
hyperbolic area (Theorem 9.1.3 in Beardon ), we have
${\text{area}(D_{0})=\text{area}(P)}$ and conclude ${D_{0}=P}$.

Now, since the elements of $[v]$ contained in $P$, which are just the $v_{i}$,
are the closest $v$-images to $0$ and lie a distance $R$ from $0$, we have
$\delta^{\star}([0],[v])=R$. Therefore ${\mathscr{D}_{g}\geq R}$.

######  Proposition 1.

Let ${\phi}$ be any isometry of $\mathbb{D}$ that maps the fundamental polygon
to any of its dual polygons. Then the projection
${\phi^{\star}:S_{g}\rightarrow S_{g}}$ given by

$\phi^{\star}([z])=[\phi(z)]$ (20)

is well-defined and is an isometric automorphism of $S_{g}$. In particular,
dual polygons are fundamental domains.

Proof: First we will show the rotational symmetry of the quotient surface. It
suffices to show that ${\theta\mathscr{F}_{g}\theta^{-1}=\mathscr{F}_{g}}$ by
the argument given in Sec. 3, where ${\theta}$ is the counterclockwise
rotation of ${\mathbb{D}}$ by ${\pi/2g}$. We will use the property of the
generators of ${\mathscr{F}_{g}}$ that ${\theta t_{i}\theta^{-1}=t_{i+1}}$ for
all ${i}$, and the fact which follows from elementary group theory that
${\theta\mathscr{F}_{g}\theta^{-1}=\mathscr{F}_{g}}$ if and only if
${\theta\mathscr{F}_{g}\theta^{-1}\subseteq\mathscr{F}_{g}}$. We further note
that it suffices to show ${\theta t_{i}\theta^{-1}\subseteq\mathscr{F}_{g}}$
for ${\forall i}$, since this implies for ${\forall
f=t_{i_{1}}t_{i_{2}}\cdot\cdot\cdot t_{i_{k}}\in\mathscr{F}_{g}}$ that

$\theta f\theta^{-1}=\theta t_{i_{1}}t_{i_{2}}\cdot\cdot\cdot
t_{i_{k}}\theta^{-1}=(\theta t_{i_{1}}\theta^{-1})(\theta
t_{i_{1}}\theta^{-1})\cdot\cdot\cdot(\theta
t_{i_{1}}\theta^{-1})\in\mathscr{F}_{g}.$ (21)

For an arbitrary generator ${t_{i}\in\mathscr{F}_{g}}$ we have

$\theta t_{i}\theta^{-1}=t_{i+1}\in\mathscr{F}_{g},$ (22)

and therefore ${\theta\mathscr{F}_{g}\theta^{-1}=\mathscr{F}_{g}}$. It follows
that the isometry ${\theta^{\star}}$, given by
${\theta^{\star}([p])=[\theta(p)]}$ for points ${[p]}$ on the quotient
surface, is a well-defined isometry and therefore a symmetry of the quotient
surface.

Next, we will show the same for ${\phi^{\star}}$. Suppose that ${\phi}$ maps
the fundamental polygon to the dual polygon ${D}$ centered at the vertex
${v}$. Then for any generator ${t_{i}}$, the map ${t_{i}^{\prime}=\phi
t_{i}\phi^{-1}}$ maps ${D}$ to one of its neighboring dual polygons centered
at vertex ${v^{\prime}}$, such that ${t^{\prime}(v)=v^{\prime}}$ and
${t^{\prime}}$ fixes the geodesic line between ${v}$ and ${v^{\prime}}$.

Now, choose ${h\in\mathscr{F}_{g}}$ that maps ${v}$ to ${v^{\prime}}$. We will
be done if we can show ${t^{\prime}=h}$, since this implies
${t^{\prime}\in\mathscr{F}_{g}}$ as desired. Since ${h}$ is orientation-
preserving, we can write it as a product of ${t^{\prime}}$ and an arbitrary
rotation by ${\alpha}$ around ${v^{\prime}}$. Then ${h^{-1}}$ is the product
of ${(t^{\prime})^{-1}}$ and a rotation by ${-\alpha}$ around ${v}$. Pick one
of the representatives of the fundamental polygon that has
${\overline{vv^{\prime}}}$ as an edge, such that the directed line segment
${\overline{vv^{\prime}}}$ is oriented counterclockwise along the polygon
boundary. Then by the rotational symmetry of ${\mathscr{F}_{g}}$, every
similar element of ${\mathscr{F}_{g}}$ that maps a vertex to its neighbor in
the counterclockwise direction has the same form, a translation followed by a
rotation by ${\alpha}$. Similarly, such maps in the clockwise direction are
translations followed by rotations by ${-\alpha}$.

However, we could have chosen the other polygon that has edge
${\overline{vv^{\prime}}}$ in the clockwise orientation, and drawn the same
conclusions above with the orientations reversed. So we are forced to conclude
${\alpha=-\alpha}$, thus either ${\alpha=0}$ or ${\alpha=\pi}$. But ${h}$
cannot have fixed points, and if ${\alpha=\pi}$ then ${h}$ fixes the midpoint
between ${v}$ and ${v^{\prime}}$, hence ${\alpha=0}$, so ${t^{\prime}=h}$.
Therefore ${\phi\mathscr{F}\phi^{-1}=\mathscr{F}}$, so ${\phi^{\star}}$ is a
symmetry of the quotient surface.

######  Proposition 2.

To prove Theorem 1, it suffices to consider only the pairs of points $z$ and
$w$ satisfying the following conditions:

  1. 1.

$z$ lies in triangle ${T=\Delta 0v_{1}v_{2}}$, Fig. 3,

  2. 2.

$d_{0}(z)\geq d_{v}(w)$,

  3. 3.

$d_{v}(w)\leq s/2$.

Proof: For pairs ${z,w}$ that do not meet at least one of the conditions
above, our strategy will be to find another pair ${z^{\prime},w^{\prime}}$
that do meet the conditions, such that
${\delta(z,w)=\delta(z^{\prime},w^{\prime})}$. Then the diameter of ${S_{g}}$,
which is the maximum of ${\delta}$ over all ${z,w}$, is equal to the maximum
of ${\delta}$ over just the ${z,w}$ which satisfy the conditions above, so it
will suffice to consider only those pairs.

Figure 4: Division of the fundamental polygon into ${8g}$ isosceles triangles
for ${g=2}$.

Starting with an arbitrary pair ${z,w}$, let $\phi$ be any of the isometries
from the fundamental polygon to a dual polygon, and $\phi^{\star}$ the
corresponding isometry of the quotient surface. Note that $\phi^{\star}$ takes
${[0]\rightarrow[v]}$ and ${[v]\rightarrow[0]}$. Therefore
${d_{0}(\phi(p))=d_{v}(p)}$ and ${d_{v}(\phi(p))=d_{0}(p)}$ for points
${p\in\mathbb{D}}$. Now, for the given $z,w$ consider the four quantities
${d_{0}(z),d_{v}(z),d_{0}(w),d_{v}(w)}$. We can divide the fundamental polygon
into ${8g}$ congruent isosceles triangles (Fig. 4) each with one vertex at
${0}$, one vertex at some ${v_{i}}$, and one vertex at the midpoint of an edge
of the polygon, and side lengths ${R,s/2,s/2}$. Since ${z}$ lies in one of
these isosceles triangles, we have ${d_{0}(z)+d_{v}(z)\leq s/2+s/2=s}$, and
similarly for ${w}$. It follows that

$M=\min\\{d_{0}(z),d_{v}(z),d_{0}(w),d_{v}(w)\\}\leq s/2.$ (23)

We now consider four cases, and in each case construct
${z^{\prime},w^{\prime}}$ for which conditions (2) and (3) are satisfied and
${\delta(z^{\prime},w^{\prime})=\delta(z,w)}$.

  1. 1.

${M=d_{0}(z)}$. Let ${z^{\prime}=\phi(w)}$ and ${w^{\prime}=\phi(z)}$. Then

$\displaystyle d_{0}(z^{\prime})=d_{v}(w)$ $\displaystyle\geq
d_{0}(z)=d_{v}(w^{\prime})\text{ and}$ (24) $\displaystyle
d_{v}(w^{\prime})=d_{0}(z)$ $\displaystyle\leq s/2.$ (25)

  2. 2.

${M=d_{v}(z)}$. Let ${z^{\prime}=w}$ and ${w^{\prime}=z}$. Then

$\displaystyle d_{0}(z^{\prime})=d_{0}(w)$ $\displaystyle\geq
d_{v}(z)=d_{v}(w^{\prime})\text{ and}$ (26) $\displaystyle
d_{v}(w^{\prime})=d_{v}(z)$ $\displaystyle\leq s/2.$ (27)

  3. 3.

${M=d_{0}(w)}$. Let ${z^{\prime}=\phi(z)}$ and ${w^{\prime}=\phi(w)}$. Then

$\displaystyle d_{0}(z^{\prime})=d_{v}(z)$ $\displaystyle\geq
d_{0}(w)=d_{v}(w^{\prime})\text{ and}$ (28) $\displaystyle
d_{v}(w^{\prime})=d_{0}(w)$ $\displaystyle\leq s/2.$ (29)

  4. 4.

${M=d_{v}(w)}$. Let ${z^{\prime}=z}$ and ${w^{\prime}=w}$. Then

$\displaystyle d_{0}(z^{\prime})=d_{0}(z)$ $\displaystyle\geq
d_{v}(w)=d_{v}(w^{\prime})\text{ and}$ (30) $\displaystyle
d_{v}(w^{\prime})=d_{v}(w)$ $\displaystyle\leq s/2.$ (31)

We have already shown the rotational symmetry of ${S_{g}}$ in the proof of
Prop. 1. Starting with $z^{\prime},w^{\prime}$ and rotating a sufficient
number of times by ${\pi/2g}$, we can place ${z^{\prime}}$ in $T$.
Furthermore, since rotations map vertices to vertices and fix ${0}$, they
preserve the quantities
${d_{0}(z^{\prime}),d_{v}(z^{\prime}),d_{0}(w^{\prime}),d_{v}(w^{\prime})}$,
so the rotated ${z^{\prime},w^{\prime}}$ now satisfy all three conditions.

######  Theorem 3.

Given $w$ for which $d_{v}(w)\leq s/2$, the function

$f(z)=\min\left[\delta(z,w_{1}),\delta(z,w_{2})\right]$

attains its global maximum over the domain $\Omega_{w}$ on its boundary
${\partial\Omega_{w}}$.

Proof: Since ${f(z)}$ is continuous and bounded on the compact region
${\Omega_{w}\cup\partial\Omega_{w}}$, $f(z)$ attains a global maximum on this
region.

Let $M$ denote the midline of $w_{1}$ and $w_{2}$ (the locus of points
equidistant from $w_{1}$ and $w_{2}$, also the perpendicular bisector of the
geodesic segment $\overline{w_{1}w_{2}}$). Then $M$ divides $\Omega_{w}$ into
some set of open regions, $\\{S_{i}\\}$. We will first show that for any ${i}$
and ${z_{0}\in S_{i}}$, $z_{0}$ cannot be a global maximum of $f(z)$.

Inside ${S_{i}}$ we have ${\delta(z,w_{1})\neq\delta(z,w_{2})}$, and therefore
either ${\delta(z,w_{1})<\delta(z,w_{2})}$ or
${\delta(z,w_{2})<\delta(z,w_{1})}$. Assume without loss of generality that
${\delta(z,w_{1})<\delta(z,w_{2})}$. Then ${f(z_{0})=\delta(z_{0},w_{1})}$ and
${f(z)=\delta(z,w_{1})}$ in some open set around $z_{0}$. Therefore, we can
always increase the value of $f$ by moving $z_{0}$ in some suitable direction
(away from $w_{1}$).

It remains to consider ${z_{0}\in M\cap\Omega_{w}}$. ${M\cap\Omega_{w}}$ is a
union of segments ${\\{M_{i}\\}}$ which lie in $\Omega_{w}$ and have endpoints
on $\partial\Omega_{w}$. Take ${z_{0}\in M_{i}}$. In this case we also have
${f(z_{0})=\delta(z_{0},w_{1})}$. It suffices to show that $\delta(z,w_{1})$
is maximized at one of the endpoints of $M_{i}$. However, this is a well-known
result on the distance from points to line segments (see Theorem 2.2 in Ramsay
). Since these endpoints are on $\partial\Omega_{w}$, we have shown that the
global maximum of ${f(z)}$ is attained on $\partial\Omega_{w}$.

######  Theorem 4.

Given $w$ for which $d_{v}(w)\leq s/2$, for ${\forall z\in\partial\Omega_{w}}$
we have

$\min\left[\delta(z,w_{1}),\delta(z,w_{2})\right]\leq R.$

Proof: We abbreviate ${a=d_{0}(z)}$, ${b=d_{v}(w)}$,
${R^{\prime}=\tanh\left(\frac{R}{2}\right)}$, and
${s^{\prime}=\tanh\left(\frac{s}{4}\right)}$.

CASE I: We look at the radial segments of ${\partial\Omega_{w}}$ first, see
Fig. 3, right. For ${z}$ on ${\overline{0v_{0}}}$, consider the triangle with
vertices ${z}$, ${v_{0}}$, and ${w_{1}}$ and side lengths ${b}$,
${\delta(z,w_{1})}$, and ${R-a}$, respectively. By the triangle inequality,

$\displaystyle b+(R-a)\geq\delta(z,w_{1})\Rightarrow$
$\displaystyle\delta(z,w_{1})\leq R-(a-b)\leq R,$ (32)

and a similar argument for ${z}$ on ${\overline{0v_{2}}}$ gives
${\delta(z,w_{2})\leq R}$, so that in either case
${\min\left[\delta(z,w_{1}),\delta(z,w_{2})\right]\leq R}$.

Figure 5: Illustration of ${z}$, ${w_{1}}$, and ${w_{2}}$, and angles
${\phi}$, ${\theta_{1}}$, ${\theta_{2}}$.

CASE II: Next we consider the circular arc of radius ${b}$ centered at the
origin (Fig. 3, right). For ${z}$ on this arc we have ${a=b}$. Let ${\phi}$
denote the angle between ${\overline{0z}}$ and ${\overline{0v_{1}}}$, so that
${\phi\in\left[0,\frac{\pi}{2g}\right]}$. Let ${\theta_{1}}$ denote the angle
between ${\overline{v_{1}w_{1}}}$ and ${\overline{v_{1}v_{2}}}$, and
${\theta_{2}}$ denote the angle between ${\overline{v_{2}w_{2}}}$ and the
extension of ${\overline{v_{1}v_{2}}}$, Fig. 5.

We have the constraint ${\theta_{1}=\theta_{2}}$, i.e. $w_{1}$ and $w_{2}$
make the same angle with the axis ${\overline{v_{1}v_{0}}}$, which is a
consequence of Prop. 1. Now, we assume that ${\delta(z,w_{1})>R}$ and
${\delta(z,w_{2})>R}$ and seek a contradiction. Using the hyperbolic sine and
cosine theorems to calculate the distances, we get after some algebra

$\displaystyle\delta(z,w_{1})>R\Leftrightarrow$
$\displaystyle\left(1+R^{\prime^{2}}\right)-2R^{\prime}\coth
a\left[\cos\left(\theta_{1}+\frac{\pi}{4g}\right)+\cos\phi\right]+$
$\displaystyle\cos\left(\theta_{1}+\phi+\frac{\pi}{4g}\right)+R^{\prime^{2}}\cos\left(\theta_{1}-\phi+\frac{\pi}{4g}\right)>0,$
(33)

and a similar equation arises from ${\delta(z,w_{2})>R}$. Solving Eq. (4)
yields

$\frac{\cos\left(\frac{\theta_{1}+\phi}{2}+\frac{\pi}{8g}\right)}{\sin\left(\frac{\theta_{1}-\phi}{2}+\frac{\pi}{8g}\right)}\in\left(-\infty,R^{\prime}\tanh\left(\frac{a}{2}\right)\right)\cup\left(\frac{R^{\prime}}{\tanh\left(\frac{a}{2}\right)},\infty\right).$
(34)

Notice that ${R^{\prime}\tanh\left(\frac{a}{2}\right)}$ is an increasing
function of ${a}$, while ${\frac{R^{\prime}}{\tanh\left(\frac{a}{2}\right)}}$
is a decreasing function of ${a}$, so it suffices to consider the case where
${a}$ is maximal, ${a=\frac{s}{2}}$. Plugging this in gives

$\frac{\cos\left(\frac{\theta_{1}+\phi}{2}+\frac{\pi}{8g}\right)}{\sin\left(\frac{\theta_{1}-\phi}{2}+\frac{\pi}{8g}\right)}\in\left(-\infty,R^{\prime}s^{\prime}\right)\cup\left(\frac{R^{\prime}}{s^{\prime}},\infty\right).$
(35)

Solving for ${\alpha=\theta_{1}-\phi+\frac{\pi}{4g}}$ gives

$\tan\left(\frac{\alpha}{2}\right)\in\left(\frac{\cos\phi-R^{\prime}/s^{\prime}}{\sin\phi},\frac{\cos\phi-R^{\prime}s^{\prime}}{\sin\phi}\right).$
(36)

A similar analysis of the second inequality, ${\delta(z,w_{2})>R}$ using
${\theta_{1}=\theta_{2}}$ gives

$\displaystyle\tan\left(\frac{\alpha}{2}\right)\in$
$\displaystyle\left(-\infty,\frac{\sin\left(\frac{\pi}{2g}-\phi\right)}{\cos\left(\frac{\pi}{2g}-\phi\right)-R^{\prime}/s^{\prime}}\right)\cup$
$\displaystyle\left(\frac{\sin\left(\frac{\pi}{2g}-\phi\right)}{\cos\left(\frac{\pi}{2g}-\phi\right)-R^{\prime}s^{\prime}},\infty\right).$
(37)

However, since the inequalities

$\frac{\cos\phi-R^{\prime}/s^{\prime}}{\sin\phi}\leq\frac{\sin\left(\frac{\pi}{2g}-\phi\right)}{\cos\left(\frac{\pi}{2g}-\phi\right)-R^{\prime}/s^{\prime}}$
(38)

and

$\frac{\cos\phi-R^{\prime}s^{\prime}}{\sin\phi}\geq\frac{\sin\left(\frac{\pi}{2g}-\phi\right)}{\cos\left(\frac{\pi}{2g}-\phi\right)-R^{\prime}s^{\prime}}$
(39)

have no solutions in ${\phi\in\left[0,\frac{\pi}{2g}\right]}$, the sets in
Eqs. (36,4) are disjoint, and we conclude that the system

$\begin{cases}\delta(z,w_{1})>R\\\ \delta(z,w_{2})>R\end{cases}$ (40)

has no solutions, so ${\min\left[\delta(z,w_{1}),\delta(z,w_{2})\right]\leq
R}$.

CASE III: Finally, we consider ${z}$ on the edge ${\overline{v_{1}v_{2}}}$.
Let ${x=\delta(z,v_{1})}$ and ${\alpha=\angle v_{2}v_{1}w_{1}}$. Applying the
first hyperbolic law of cosines to triangle ${\Delta zv_{1}w_{1}}$ gives

$\cosh\left(\delta(z,w_{1})\right)=\cosh b\cosh x-\sinh b\sinh x\cos\alpha.$
(41)

Assuming for the sake of contradiction that ${\delta(z,w_{1})>R}$ and
${\delta(z,w_{2})>R}$, then from the first inequality we find

$\cosh b\cosh x-\sinh b\sinh x\cos\alpha>\cosh R,$ (42)

which implies

$\cos\alpha<\frac{\cosh b\cosh x-\cosh R}{\sinh b\sinh x}.$ (43)

Similarly, the second inequality ${\delta(z,w_{2})>R}$ yields

$\cos\alpha>\frac{\cosh R-\cosh b\cosh(s-x)}{\sinh b\sinh(s-x)}.$ (44)

However, for inequalities (43) and (44) to have solutions in ${\alpha}$, we
must have

$\frac{\cosh b\cosh x-\cosh R}{\sinh b\sinh x}>\frac{\cosh R-\cosh
b\cosh(s-x)}{\sinh b\sinh(s-x)}.$ (45)

Simplifying yields

$\cosh\left(\frac{s}{2}\right)\cosh\left(x-\frac{s}{2}\right)<\cosh b.$ (46)

Notice that the right hand side is an increasing function of ${b}$, thus it
suffices to disprove the inequality in the case where ${b}$ is maximal,
${b=\frac{s}{2}}$. Plugging this in and simplifying the resulting inequality
gives

$\cosh\left(x-\frac{s}{2}\right)<1,$ (47)

which has no solutions in ${x}$. Therefore our assumption was false, so
${\min\left[\delta(z,w_{1}),\delta(z,w_{2})\right]\leq R}$.

######  Theorem 1.

${\mathscr{D}_{g}=R}$.

Proof: We first show that for ${\forall[z],[w]}$,

$\delta^{\star}([z],[w])\leq R.$ (48)

By Prop. 2, it suffices to consider the case ${z\in\Delta 0v_{1}v_{2}}$,
${d_{0}(z)\geq d_{v}(w)}$, ${d_{v}(w)\leq s/2}$. Then by Theorem 3, for all
such $w$ the function ${f(z)=\delta^{\star}([z],[w])}$ defined on the domain
${\Omega_{w}}$ attains a global maximum $M$ on the boundary
${\partial\Omega_{w}}$, and by Theorem 4, $M\leq R$. This proves Eq. (48), and
therefore $\mathscr{D}_{g}\leq R$. But Theorem 2 says that
$\mathscr{D}_{g}\geq R$. We conclude that $\mathscr{D}_{g}=R$.

ACKNOWLEDGEMENTS

We thank Florent Balacheff, Maxime Fortier Bourque, Antonio Costa, Benson
Farb, Svetlana Katok, Gabor Lippner, Marissa Loving, Curtis McMullen, Hugo
Parlier, John Ratcliffe, and Chaitanya Tappu for useful discussions and
suggestions. This work was supported by NSF grant No. IIS-1741355.

## References

  *   * (1) Seifert, H., Threlfall, W.: Lehrbuch der Topologie: Von H. Seifert und W. Threlfall. B.G. Teubner, Leipzig (1934). https://books.google.com/books?id=bRHRxQEACAAJ
  * (2) Chang, F.-R.: On the diameters of compact Riemann surfaces. Proceedings of the American Mathematical Society 65(2), 274–276 (1977). https://doi.org/10.2307/2041905
  * (3) Balacheff, F., Despré, V., Parlier, H.: Systoles and diameters of hyperbolic surfaces. Kyoto Journal of Mathematics 1(1), 1–12 (2023). https://doi.org/10.1215/21562261-2022-0040
  * (4) Budzinski, T., Curien, N., Petri, B.: On the minimal diameter of closed hyperbolic surfaces. Duke Mathematical Journal 170(2), 365–377 (2021). https://doi.org/10.48550/arXiv.1909.12283
  * (5) Bavard, C.: Disques extrémaux et surfaces modulaires. Annales de la Faculté des Sciences de Toulouse. Mathématiques. Série VI 5(2), 191–202 (1996). https://doi.org/10.5802/afst.827
  * (6) Bolza, O.: On binary sextics with linear transformations into themselves. American Journal of Mathematics 10(1), 47–70 (1887). https://doi.org/%****␣diameter_of_compact_riemann_surfaces.bbl␣Line␣125␣****10.2307/2369402
  * (7) Hadamard, J.: Les Surfaces à Courbures Opposées et Leurs Lignes Géodésiques. Journal de Mathématiques Pures et Appliquées 4, 27–74, (1898). http://eudml.org/doc/235168
  * (8) Jenni, F.: Über Den Ersten Eigenwert des Laplace-Operators Auf Ausgewählten Beispielen Kompakter Riemannscher Flächen. Commentarii Mathematici Helvetici 59(2), 193–203, (1984). https://eudml.org/doc/139972
  * (9) Ebbens, M., Iordanov, I., Teillaud, M., Vegter, G.: Delaunay triangulations of generalized Bolza surfaces. Journal of Computational Geometry 13(1), 125–177 (2022). https://doi.org/10.20382/jocg.v13i1a5
  * (10) Wiman, A.: Über die Hyperelliptischen Curven und Diejenigen Von Geschlechte ${p=3}$, welche eindeutige Tiansformationen in sich zulassen. Bihang till Kongl. Svenska vetenskaps-akademiens handlingar 21, 1–23, (1895). https://babel.hathitrust.org/cgi/pt?id=msu.31293024926804&view=1up&seq=11
  * (11) Bujalance, E., Conder, M.D., Costa, A.F., Izquierdo, M.: On regular dessins d’enfants with 4g automorphisms and a curve of wiman. Algebraic Curves and Their Applications, Amer. Math. Soc.,[Providence], RI 724, 225–233 (2019). https://doi.org/10.1090/conm/724/14592
  * (12) Bujalance, E., Costa, A.F., Izquierdo, M.: On Riemann surfaces of genus g with 4g automorphisms. Topology and its Applications 218, 1–18 (2017). https://doi.org/10.1016/j.topol.2016.12.013
  * (13) Nabutovsky, A., Weinberger, S.: The fractal nature of Riem/Diff I. Geometriae Dedicata 101(1), 1–54 (2003). https://doi.org/10.1023/A:1026358815492
  * (14) Beardon, A.F.: The Geometry of Discrete Groups. Springer, New York (2012). https://doi.org/10.1007/978-1-4612-1146-4
  * (15) Cheng, X., Zhou, D.: First eigenvalue estimate on Riemannian manifolds. Hokkaido Mathematical Journal 24, 453–472 (1995). https://doi.org/10.14492/hokmj/1380892604
  * (16) Cheng, S.Y.: Eigenvalue comparison theorems and its geometric applications. Mathematische Zeitschrift 143(3), 289–297 (1975). https://doi.org/10.1007/BF01214381
  * (17) Cheeger, J.: The relation between the laplacian and the diameter for manifolds of non-negative curvature. Archiv der Mathematik 19, 558–560 (1968). https://doi.org/10.1007/BF01898781
  * (18) Aurich, R., Steiner, F.: On the periodic orbits of a strongly chaotic system. Physica D Nonlinear Phenomena 32, 451–460 (1988). https://doi.org/10.1016/0167-2789(88)90068-1
  * (19) Katok, S.: Fuchsian Groups. University of Chicago Press, Chicago (1992). https://doi.org/10.2307/3619761
  * (20) Abrams, A., Katok, S., Ugarcovici, I.: Rigidity and flexibility of entropies of boundary maps associated to Fuchsian groups. Mathematics Research Reports 3, 1–19 (2022). https://doi.org/10.5802/mrr.10
  * (21) Ramsay, A., Richtmyer, R.D.: Introduction to Hyperbolic Geometry. Springer, New York (1995). https://doi.org/10.1007/978-1-4757-5585-5


# ShapeClipper: Scalable 3D Shape Learning from Single-View Images
via Geometric and CLIP-based Consistency

Zixuan Huang1 Varun Jampani2 Anh Thai1 Yuanzhen Li2
Stefan Stojanov1 James M. Rehg1
1Georgia Institute of Technology, 2Google Research

###### Abstract

We present ShapeClipper, a novel method that reconstructs 3D object shapes
from real-world single-view RGB images. Instead of relying on laborious 3D,
multi-view or camera pose annotation, ShapeClipper learns shape reconstruction
from a set of single-view segmented images. The key idea is to facilitate
shape learning via CLIP-based shape consistency, where we encourage objects
with similar CLIP encodings to share similar shapes. We also leverage off-the-
shelf normals as an additional geometric constraint so the model can learn
better bottom-up reasoning of detailed surface geometry. These two novel
consistency constraints, when used to regularize our model, improve its
ability to learn both global shape structure and local geometric details. We
evaluate our method over three challenging real-world datasets, Pix3D,
Pascal3D+, and OpenImages, where we achieve superior performance over state-
of-the-art methods.111Project website at:
https://zixuanh.com/projects/shapeclipper.html

Figure 1: We propose a method that reconstructs 3D object shape from single
view real-world images. Our method learns high-quality reconstruction through
_single-view supervision without known viewpoint_ and can reconstruct shapes
of various objects.

##  1 Introduction

How can we learn 3D shape reconstruction from real-world images in a scalable
way? Recent works achieved impressive results via learning-based approaches
either with 3D [4, 9, 11, 28, 41, 36, 49, 45, 43, 44, 48, 53] or multi-view
supervision [50, 42, 51, 23, 16, 19, 25, 32]. However, such supervised
techniques cannot be easily applied to real-world scenarios, because it is
expensive to obtain 3D or multi-view supervision at a large scale. To address
this limitation, recent works relax the requirement for 3D or multi-view
supervision [17, 10, 22, 8, 14, 55, 31, 18, 46, 24, 15, 1, 29, 38, 57]. These
works only require single-view self-supervision, with some additionally using
expensive viewpoint annotations [17, 18, 24, 38, 57]. Despite this significant
progress, most methods still suffer from two major limitations: 1) Incorrect
top-down reasoning, where the model only explains the input view but does not
accurately reconstruct the full 3D object shape; 2) Failed bottom-up
reasoning, where the model cannot capture low-level geometric details such as
concavities. How can we address these limitations while also remaining
scalable to a wide range of object types?

To improve top-down reasoning, our inspiration comes from the recent success
of large-scale image-text modeling. The most successful image-text models such
as CLIP [33] are trained on a vast corpus of captioned images and are able to
extract fine-grained semantic features that correlate well with the language
descriptions. CLIP further demonstrates a great generalization ability to
images across various domains. Can we leverage such a powerful and
generalizable model to learn 3D reconstruction in a real-world scenario?

Figure 2: CLIP-based semantic neighbors. Samples that have similar CLIP
encodings often have similar shapes. Note the viewpoint variability in the
neighbors.

We observe that natural language descriptions of images often contain
geometry-related information (_e.g_. a round speaker, a long bench) and many
nouns by themselves have characteristic shape properties (_e.g_. “desks”
usually have four legs, and “benches” normally include a flat surface).
Motivated by this intrinsic connection between object shapes and language-
based semantics, we examine the latent space of CLIP’s visual encoder. In our
study, we find (via k-NN queries) that objects with similar CLIP embeddings
usually share similar shapes (see Fig. 2 for an example). Another key
characteristic we identify with CLIP embeddings is that they have some
robustness to viewpoint changes, meaning that changes in viewpoint generally
do not produce drastic changes in CLIP embeddings.

Figure 3: Semantic-based Shape Consistency (SSC) Constraint. We find the
semantic neighbors of the input image across the training set and use these
neighbors to regularize the shape learning.

Inspired by these findings, we propose to learn shapes using a semantic-based
shape consistency (SSC) constraint using CLIP. Specifically, we use CLIP’s
semantic encodings as guidance to form pseudo multi-view image sets. For each
image in the training set, we extract its CLIP embedding and find images with
the most similar semantics across the training set. We then leverage these
retrieved images as additional supervision to the input view, as illustrated
in Fig. 3. This approach greatly benefits global shape understanding, because
each predicted shape is required to simultaneously explain a set of images
instead of only explaining the single input image.

On the other hand, we address the limitation of poor bottom-up geometric
reasoning by constraining the surface normals of the predicted shapes. Common
failure cases include noisy surface reconstruction and failed concavity
modeling, which are extremely hard to learn even with multi-view supervision.
Inspired by the recent success of large-scale 2.5D estimation that generalizes
to various scenes [7, 35, 34], we propose to use off-the-shelf surface normals
as additional geometric supervision for our task. However, unlike scenes, off-
the-shelf normals for object-centric images are much noisier due to occlusion,
truncation, and domain gaps. To mitigate this issue, we introduce a noise-
tolerant optimization process via outlier dropout, which stabilizes the
training and improves the overall reconstruction performance.

Overall, our contributions are threefold:

  * •

We propose a novel CLIP-based shape consistency regularization that greatly
facilitates the top-down understanding of object shapes.

  * •

We successfully leverage off-the-shelf geometry cues to improve single-view
object shape reconstruction for the first time and handle noise effectively.

  * •

We perform extensive experiments across 3 different real-world datasets and
demonstrate state-of-the-art performance.

##  2 Related Work

There has been an emerging interest in 3D object shape reconstruction from
images via learning-based approaches. Our work focuses on learning single-view
shape reconstruction with limited supervision on real-world images, where the
training set only contains a single view per object instance. We briefly
survey the relevant literature on single image shape reconstruction using both
fully-supervised and weakly-supervised approaches.

Single-View Supervision. Most closely related to this paper are works that
learn 3D shape reconstruction through supervision from single-view images [17,
10, 22, 14, 13, 55, 31, 18, 46, 24, 57, 15, 1, 29]. These works can be
organized as in Tab. 1 and largely differ in their choice of 1) shape
representation (e.g. implicit SDF vs explicit mesh); 2) known vs. unknown
viewpoint supervision; 3) large-scale evaluation on various real-world
objects. We are one of the earliest works to demonstrate the feasibility of
single-view learning of an implicit SDF representation upon diverse real-world
images under unknown viewpoints.

Table 1: Single-view supervised methods for object shape reconstruction. M: mesh, V: voxel, P: pointcloud, D: depth, O: occupancy function, S: signed distance function, Diverse R-Res.: real-world results on diverse categories. Model | [17] | [18] | [24] | [38] | [57] | [10] | [22] | [14] | [13] | [31] | [46] | [29] | [15] | [55] | [1] | Ours
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
3D Rep. | M | M | S | M | M | M | M | V | M | P | D | M | S | M | O | S
Viewpoint Free | - | - | - | - | - | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓
Diverse R-Res. | - | - | - | - | - | - | - | - | - | - | - | - | - | ✓ | ✓ | ✓

Within this body of work, SSMP [55], Cat3D [15], and SS3D [1] are the most
closely related ones given their large-scale evaluation, which we describe in
details below.

SSMP [55] is the earliest work that shows success of shape learning via only
single-view supervision on large-scale real-world data. A key property of this
method is adversarial regularization during training, which can make training
unstable. Thus it is hard for SSMP to learn reconstruction on categories with
complex shapes or textures. In contrast, our method leverages the SSC and
geometric constraints which are more stable and result in superior performance
over SSMP across various objects.

Similar to our method, Cat3D [15] explores semantic regularization for
implicit shape learning. In contrast, their semantic regularization is based
on category labels, which fails on categories with significant intra-category
shape variations. Moreover, Cat3D relies on unstable adversarial
regularization which has similar drawbacks as SSMP [55] and is only successful
on a few real-world categories.

SS3D [1] proposes a 3-step learning pipeline for scalable learning of shapes,
which includes synthetic data (e.g. ShapeNet [3]) pretraining. This step plays
a crucial role as it provides the necessary initialization for the camera
multiplex optimization. Unlike SS3D, synthetic pretraining is not a hard
constraint for our method—we demonstrate success on Pix3D [39] without any
synthetic pretraining. On the other hand, SS3D does not explore the usage of
semantic and geometric consistency. As a result, our model captures both
global structures and local surfaces more accurately than SS3D and outperforms
SS3D quantitatively.

Shape Supervision. Instead of using image supervision, many prior works use
explicit 3D geometric supervision and achieve great reconstruction results [4,
9, 11, 28, 41, 36, 49, 45, 43, 44, 48, 53]. Nevertheless, the assumption of 3D
supervision is not yet practical on a large scale. To make the learning more
scalable, subsequent works leverage multi-view images as supervision and
employ differentiable rendering as the core technique. Specifically,
differentiable rendering allows images and masks to be rendered from 3D assets
differentiably and thus the multi-view reprojection loss can effectively carve
the reconstructed shape. These methods can be classified based on their
representation of shape, including voxels [50, 42, 51], pointclouds [23, 16],
meshes [19, 25] and implicit representations [32]. Compared to these works, a
major benefit of our method is scalability, as our model can be trained using
single-view real-world images.

##  3 Method

In this section, we first present an overview of our model in Sec. 3.1, and
then introduce our proposed SSC and geometric constraints in Sec. 3.2 and Sec.
3.3. Finally, we present implementation details in Sec. 3.4.

###  3.1 Overview

Given a collection of $n$ images segmented with foreground masks
$\\{I_{i}\in\mathbb{R}^{h\times w\times 3},M_{i}\in\mathbb{R}^{h\times w\times
1}\\}_{i=1}^{n}$, we aim to learn a single-view 3D reconstruction model
without relying on 3D, viewpoint, or multi-view supervision of these images.
The shape representation of our model is an implicit SDF function, represented
by a multi-layer perceptron (MLP) conditioned on the input image.
Specifically, our model consists of four submodules (see Fig. 4 for an
overview) as described below.

Figure 4: Network overview. Given the input image, the encoder
$\boldsymbol{E}$ infers the shape latent code $\boldsymbol{s}$ and texture
latent code $\boldsymbol{t}$. By conditioning these two codes upon shape MLP
$\boldsymbol{f_{S}}$ and texture MLP $\boldsymbol{f_{T}}$, we obtain the shape
and texture reconstruction of the input object. On the other hand, the
viewpoint estimator $\boldsymbol{V}$ estimates the input viewpoint
$\boldsymbol{v}$. The differentiable volume renderer $\boldsymbol{R}$ then
renders shape and texture fields under the estimated viewpoint, so that we can
compute the reconstruction loss $\mathcal{L}_{I}$ and $\mathcal{L}_{M}$. We
further leverage our SSC and geometric constraints, $\mathcal{L}_{SSC}$ and
$\mathcal{L}_{N}$, to effectively harness the shape learning.

Image encoder. The image encoder $\boldsymbol{E}$ takes a segmented image
$I\in\mathbb{R}^{h\times w\times 3}$ as input and infers the shape latent code
$\boldsymbol{s}\in\mathbb{R}^{l}$ and the texture latent code
$\boldsymbol{t}\in\mathbb{R}^{l}$. These two codes encode the necessary
information to reconstruct the shape and texture field respectively.

Shape and texture reconstructor. Our model represents shape and texture
reconstruction with two MLPs,
$\boldsymbol{f_{S}}:\mathbb{R}^{3}\rightarrow\mathbb{R}$ and
$\boldsymbol{f_{T}}:\mathbb{R}^{3}\rightarrow\mathbb{R}^{3}$, which predict
signed distance function (SDF) and RGB values of queried 3D coordinates
respectively. The MLPs are conditioned on the latent codes, with a similar
design to VolSDF [54]. Specifically, the shape MLP $\boldsymbol{f_{S}}$ is
conditioned on $\boldsymbol{s}$ and the texture MLP $\boldsymbol{f_{T}}$ is
conditioned on $\boldsymbol{t}$.

Viewpoint estimator. The viewpoint estimator $\boldsymbol{V}$ estimates the
viewpoint of the input image with respect to the shape reconstruction.
Following [2, 15], we represent the viewpoint with the trigonometric functions
of Euler angles, i.e.
$\boldsymbol{v}=[\cos\boldsymbol{\gamma},\sin\boldsymbol{\gamma}]$ where
$\boldsymbol{\gamma}$ denotes the three Euler angles.

Differentiable renderer. We use a volume renderer $\boldsymbol{R}$ to render
the reconstructed SDF and texture fields following VolSDF [54]. In the
renderer, the SDF field is first converted into densities, and then the
densities are used together with the texture field to render the RGB and mask
in an accumulative way (via ray-marching). We refer the readers to VolSDF [54]
for more details, with the exception that we use uniform sampling instead of
error-bound based sampling. Formally, we denote the renderer as a functional,
$\boldsymbol{R}(\boldsymbol{f_{S}},\boldsymbol{f_{T}},\boldsymbol{v})$, which
maps the implicit functions and the viewpoint into image $\hat{I}$, mask
$\hat{M}$ and surface normal $\hat{N}$.

Reprojection loss. One of the major learning signals of our model comes from
the reprojection loss that compares input images with reconstructed images.
This can be achieved via the differentiable renderer. Our model first infers
shape, texture, and viewpoint of the object from the input image. The renderer
can then render them into an image reconstruction, which will be matched to
the input.

Specifically, we can denote the RGB and the mask reprojection loss for each
image as follows:

$\mathcal{L}_{I}=\|I-\hat{I}\|^{2}_{2},\ \mathcal{L}_{M}=1-IOU(M,\hat{M}),$
(1)
$IOU(M,\hat{M})=\frac{\sum_{p}M^{p}\cdot\hat{M^{p}}}{\sum_{p}M^{p}+\hat{M^{p}}-M\cdot\hat{M^{p}}}.$
(2)

Here $M^{p}$ denotes the mask value at pixel $p$.

Facilitating shape learning. When we do not have direct viewpoint or shape
supervision, simply minimizing the reprojection loss almost always leads to
degeneration. There are two major issues: 1) incorrect top-down reasoning,
where shapes can only explain the input view; 2) wrong bottom-up reasoning,
examples include the inability to infer concavity or noisy surface
reconstruction. To mitigate these issues, we propose the semantic and
geometric consistency constraints that effectively facilitate the shape
learning.

###  3.2 Semantic Constraint

Preliminary findings about CLIP. To leverage CLIP for shape regularization,
our main hypothesis is that objects with similar CLIP encodings share similar
shapes. To verify this hypothesis, we perform a study using the large-scale
fine-grained CompCars [52] dataset. This dataset contains more than 136K
images of 163 car makes with 1716 car models. We perform CLIP inference on
this dataset and compute 5-nearest neighbor for each sample based on the CLIP
embeddings. By iterating over each neighbor of all query images, we calculate
the percentage of neighbors that match their query images’ model (same car
model usually shares quite similar shapes). In our experiment, CLIP is able to
find the exact same car models for 51.2% of all the neighbors (on average 2.6
out of 5 neighbors belong to the query images’ model). We believe this is a
promising finding given 1) the large number of images and models in CompCars
and 2) the fact that different models can still have similar shapes, so the
percentage of shape matches can be higher than exact model matches. As a
comparison, the percentage of model matches for ImageNet-pretrained ViT [6] is
only 27.8%. This study verifies our hypothesis and enables us to design our
Semantic-based Shape Consistency (SSC) constraint based on CLIP.

Semantic-based Shape Consistency. The key idea of SSC is to pull instances
with similar CLIP embeddings together, so that a single shape reconstruction
can receive supervision from all these instances. In our experiments, we find
CLIP encodings have some robustness to viewpoint change (see supplement for
more details). Therefore it enables us to find additional pseudo views for
many objects, which significantly facilitate the learning of better top-down
reasoning.

We first form the per-instance clusters by performing K-nearest neighbors with
CLIP encodings. Formally, given our training set $\\{I_{i}\\}$, we extract the
CLIP encoding for each image, denoted as $\\{\boldsymbol{c}_{i}\\}$. We
calculate the cosine similarity of all pairs of encodings. With such
similarity measurement, we can query the K-nearest neighbors for any specific
input encoding $\\{\boldsymbol{c}_{i}\\}$ and identify images and masks of
these neighbors.

Figure 5: Semantic-based Shape Consistency (SSC) constraint. We improve shape
learning via the SSC constraint. The shape reconstructed for the input object
has to explain its CLIP semantic neighbor as well.

We then use these semantic neighbors to supervise the shape reconstruction as
in Fig. 5. The high-level idea is that 1) the shape reconstructed for the
input should explain neighbors’ masks and normals, and 2) when combining
input’s shape with neighbor’s texture, we should be able to render the
neighbor image.

Formally, for input $I$, we denote its shape latent code and shape MLP as
$\boldsymbol{s}$ and $\boldsymbol{f_{S}}$. Meanwhile, we sample an image and
its mask $I_{k}$ and $M_{k}$ from the semantic neighbor set
$\\{I_{k},M_{k}\\}^{K}_{k=1}$. The encoder $\boldsymbol{E}$ then predicts the
latent texture code $\boldsymbol{t}_{k}$ for $I_{k}$, which is used to
generate its texture function $\boldsymbol{f_{T}}_{k}$. We also obtain the
viewpoint prediction $\boldsymbol{v}_{k}$ through the viewpoint estimator
$\boldsymbol{V}$. We can then combine the input shape $\boldsymbol{f_{S}}$
with the neighbor texture $\boldsymbol{f_{T}}_{k}$ and viewpoint
$\boldsymbol{v}_{k}$, and render them into an image $\hat{I}^{\prime}$, a mask
$\hat{M}^{\prime}$ and a normal map $\hat{N}^{\prime}$. Namely,
$(\hat{I}^{\prime},\hat{M}^{\prime},\hat{N}^{\prime})=\boldsymbol{R}(\boldsymbol{f_{S}},\boldsymbol{f_{T}}_{k},\boldsymbol{v}_{k})$.
By replacing input maps in Eq. 1 with semantic neighbors, we can obtain the
SSC losses for this sample in the same form:

$\mathcal{L}_{SSC_{I}}=\|I_{k}-\hat{I}^{\prime}\|^{2}_{2},\
\mathcal{L}_{SSC_{M}}=1-IOU({M_{k}},\hat{M^{\prime}}).$ (3)

###  3.3 Geometric Constraint

We further propose to facilitate shape learning via geometric constraints to
encourage the model to learn better low-level geometric reasoning. The idea
here is to estimate the surface normal of our implicit shape, and make it
consistent with the surface normal prediction from off-the-shelf models. The
off-the-shelf normal estimator we use is Omnidata [7], which is a state-of-
the-art normal estimator. Recent work has also proven its effectiveness for
multi-view scene reconstruction [56].

Formally, we denote our surface normal estimation as
$\hat{N}\in\mathbb{R}^{h\times w\times 3}$ and the off-the-shelf unit normal
as $N\in\mathbb{R}^{h\times w\times 3}$. The estimated normal is calculated as
the normalized gradient of the density and aggregated via volume rendering
similar to MonoSDF [56]. Unlike the setup in MonoSDF, our normal estimation
lies in the object-centric canonical frame instead of the view-centric frame.
Therefore, we use our estimated viewpoint to rotate the off-the-shelf normal
$N$ to be in the same canonical frame as $\hat{N}$. In addition to aligning
the coordinate frames, this approach enables the viewpoint estimator to
receive additional training signals from local geometry alignment, which is a
significant benefit that naive approaches like the closed-form rotation
alignment cannot provide. After the rotation, we can then match the normals
following [7]:

$\mathcal{L}_{N}=\beta\cdot\|RN-\hat{N}\|_{1}-cos(RN,\hat{N}),$ (4)

where $R$ refers to the rotation matrix derived from the estimated viewpoint
and $cos$ denotes the cosine similarity. We set $\beta=5$ across all the
experiments.

This geometric loss is calculated at the pixel level and averaged over a
minibatch. However, unlike scene reconstruction [56], off-the-shelf normals
can be noisy for object-centric images due to inaccurate masks and domain
gaps. As a result, naively using off-the-shelf normals results in training
instability. Inspired by online hard example mining [37], we propose to
dropout off-the-shelf normals that are likely to be outliers via batchwise
ranking. Specifically, we sort the normal loss $\mathcal{L}_{N}$ within the
current minibatch and exclude a fixed percentage of high-loss pixels from the
final loss aggregation. We find that this strategy stabilizes the training and
improves the reconstruction quality overall.

Finally, we can combine the geometric constraint with the semantic constraint
by having a SSC normal loss, $\mathcal{L}_{SSC_{N}}$. This can be calculated
similarly to $\mathcal{L}_{N}$, the only difference is that we replace the
input off-the-shelf normals and rotations with the semantic neighbor’s as in
Eq. 3.

###  3.4 Implementation Details

Architecture. The image encoder we use is a ResNet34 [12], which projects the
input image into two 64-d latent vectors representing shape and texture. We
use lightweight MLPs to represent the SDF and texture fields, where the shape
MLP has 5 hidden layers of 64 neurons and the texture MLP has 3 hidden layers
of 64 neurons. The 3D coordinates are positionally encoded [27] before fed
into the MLPs. The conditioning of the MLPs is achieved via concatenation, and
the shape latent code is additionally skip-connected to the first and the
second hidden layers of the shape MLP. Following VolSDF, we condition the
texture MLP with the shape MLP’s last-layer feature as well. The
differentiable renderer we use renders the volumes by uniformly sampling 64
points along each ray.

Loss function. Our overall loss function is a summation of the reconstruction
loss and the SSC losses (with our geometric constraint included):

$\mathcal{L}_{recon}=\mathcal{L}_{I}+\lambda_{1}\mathcal{L}_{M}+\lambda_{2}\mathcal{L}_{N},$
(5)
$\mathcal{L}_{SSC}=\mathcal{L}_{SSC_{I}}+\lambda_{1}\mathcal{L}_{SSC_{M}}+\lambda_{2}\mathcal{L}_{SSC_{N}},$
(6) $\mathcal{L}=\mathcal{L}_{recon}+\mathcal{L}_{SSC}.$ (7)

We set $\lambda_{1}=0.5$ and $\lambda_{2}=0.01$ across all datasets.

Training. We use the Adam [20] optimizer with a learning rate of 0.0001 and a
batch size of 12. We did not use weight decay or learning rate scheduling.
Instead of using all pixels at once, we sample 512 rays to perform ray-
marching for each image. Our model is trained on a single NVIDIA GTX TITAN Xp
for 200 to 400 epochs depending on the dataset size, which usually takes 1 to
3 days to train. Following SS3D [1], we initialize the model by pretraining on
ShapeNet for our experiments on Pascal3D+ and OpenImages, where we compare to
SS3D. We use the commonly used symmetry constraint [30, 55] and regularize the
azimuth with a uniform prior. We further regularize the SDF field with the
eikonal loss so the gradient norm is close to 1. For Pix3D we did not use
synthetic pretraining, and instead we pretrain the shape to a sphere for a
better initialization similar to Cat3D [15].

##  4 Experiments

We present the findings of applying our method across three real-world
datasets in this section, including state-of-the-art comparison and detailed
ablations. We first introduce the datasets we use and then describe the
evaluation metrics as well as the baselines. Finally, we show detailed
experiment results on each dataset.

###  4.1 Datasets

We evaluate our method across three real-world datasets, including Pix3D [39],
Pascal3D+[47] and OpenImages [21].

Pix3D. Pix3D is a real-world 3D object dataset where each image is annotated
with a corresponding object mask, a CAD model, and the input viewpoint. This
3D information is obtained via manual alignment between shapes and images. We
use the chair category, which is the dominant category of this dataset. We
follow the 70/10/20 split of [15], resulting in 2007, 303, 584 images for
training/validating/testing respectively.

Pascal3D+. Pascal3D+ is a real-world 3D object dataset obtained similarly to
Pix3D. Compared to Pix3D, this dataset is more challenging because 1) it
includes 12 diverse categories with a high-variance viewpoint distribution,
and 2) object masks are quite noisy and some objects are occluded. We use all
categories in the ImageNet [5] subset of this dataset, with a similar split to
[24, 15], resulting in 11317, 11421 images for training and testing
respectively.

OpenImages. Unlike Pix3D and Pascal3D+, OpenImages do not contain any 3D
annotation, which prevents us from evaluating methods quantitatively. We use
20 diverse categories (more in supplement) to train our model and leverage the
occlusion scores from [55] to filter out highly occluded images. Each category
includes 1000 to 3000 images, which are split into 90/10 for training and
testing.

Figure 6: Qualitative comparison on OpenImages. Our method learns both better
global 3D structure and shape details on various categories.

###  4.2 Evaluation

To evaluate the implicit shapes, we sample SDF values with a $100^{3}$ spatial
grid and extract the 0-isosurface via Marching Cubes [26]. We further align
the coordinate frames between predicted meshes and ground truth (GT) meshes,
by transforming all meshes into the view-centric coordinate frame. We also
align the scales of these meshes using the size of their projections on the
image plane. After these steps, we can sample points from mesh surfaces and
calculate Chamfer Distance and F-score as our quantitative metrics following
[24, 11, 40, 41, 15].

Chamfer Distance. Following [15], Chamfer Distance (CD) is defined as an
average of accuracy and completeness. Given two pointclouds $S_{1}$ and
$S_{2}$, CD can be written as:

$\small d(S_{1},S_{2})=\frac{1}{2|S_{1}|}\sum_{x\in S_{1}}\min_{y\in
S_{2}}\|x-y\|_{2}+\frac{1}{2|S_{2}|}\sum_{y\in S_{2}}\min_{x\in
S_{1}}\|x-y\|_{2}$ (8)

F-score. F-score (FS@$d$) is a joint measurement of accuracy and completeness
with a given threshold $d$. Specifically, precision@$d$ is the percentage of
predicted points that have at least one GT neighbor within distance $d$.
Similarly, recall@$d$ is the percentage of ground truth points that have at
least one neighboring predicted points within distance $d$. FS@$d$ is then
calculated as the harmonic mean of precision@$d$ and recall@$d$. It can be
intuitively understood as the percentage of surface that are correctly
reconstructed.

Table 2: Quantitative results on Pix3D. Our method performs favorably to baselines and other SOTA methods.  Methods | FS@1$\uparrow$ | FS@5$\uparrow$ | FS@10$\uparrow$ | CD$\downarrow$
---|---|---|---|---
w/o $\mathcal{L}_{SSC}$ | 0.0958 | 0.4309 | 0.7093 | 0.749
w/o normal | 0.0815 | 0.3913 | 0.6982 | 0.766
w/o noise-tol | 0.1277 | 0.5319 | 0.7861 | 0.640
Ours | 0.1317 | 0.5473 | 0.8002 | 0.618
Cat3D [15] | 0.0960 | 0.4410 | 0.7262 | 0.679
SSMP [55] | 0.0948 | 0.4261 | 0.7168 | 0.707

###  4.3 Baselines

We consider three different baselines in this work, including SSMP [55], Cat3D
[15] and SS3D [1].

SSMP learns single-view supervised voxel reconstruction via adversarial
regularization.222SSMP has an optional test-time optimization to further
convert voxels into meshes. Using the authors’ public implementation of this
optimization did not improve their results in our experiments, so we use their
voxel prediction for evaluation. We compare our method to SSMP over Pix3D and
Pascal3D+.

Cat3D learns multi-class shape reconstruction without using any 3D/viewpoint
annotation. It uses a similar implicit SDF representation. We compare our
method to Cat3D over Pix3D and Pascal3D+.

SS3D learns single-view supervised implicit shape reconstruction by
pretraining on ShapeNet first. They use a per-instance camera multiplex
optimization, which is too computationally expensive for us to train their
model (based on their paper, even training on a single category-specific model
takes 64 V100 days on average). Therefore, we use their publicly available
pretrained weights instead and evaluate their method on Pascal3D+ by selecting
11 categories that SS3D has seen during training. Additionally, because SS3D
cannot predict viewpoints during inference, we use the brute-force evaluation
to evaluate shape reconstruction when comparing our method to SS3D.
Specifically, for each instance, we align the scales of the reconstructed
shape and the GT shape, and search for the rotation that leads to the lowest
Chamfer Distance. We also compare our method to SS3D on OpenImages
qualitatively.

###  4.4 Pix3D

We perform experiments on Pix3D and show quantitative and qualitative results
in Tab. 2 and Fig. 7.

Figure 7: Qualitative comparison on Pix3D. Our method learns better global 3D
structure and shape details than other baselines.

Ablation Study. We first analyze the results of ablating the techniques we
propose. In Tab. 2, ‘w/o $\mathcal{L}_{SSC}$’ refers to our model without the
CLIP-based semantic constraint, ‘w/o normal’ refers to our model without the
geometric constraint, ’w/o noise-tol’ refers to our model without outlier
dropping in the geometric constraint. Because we do not perform synthetic
pretraining on Pix3D, we find the final performance of the baselines sensitive
to the initialization. Therefore we average the quantitative results over 5
runs with different random seeds. By comparing the baselines to our final
model, we clearly see our semantic and geometric constraints improve the
reconstruction performance, and the outlier dropping benefits the shape
learning as well. In the qualitative results (Fig. 7), we find our semantic
constraint leads to better global structures, while our geometric constraint
significantly improves the reconstruction of object surfaces.

SOTA Comparison. Comparing with SOTA methods (Tab. 2, last 3 rows), we see our
approach outperforms Cat3D and SSMP significantly. Qualitatively, our approach
captures better overall shape topology and local geometric details than Cat3D
and SSMP. These results all demonstrate the effectiveness of our proposed
method. Note that for comparisons on this dataset there is no synthetic pre-
training for any methods.

###  4.5 Pascal3D+

We perform experiments on Pascal3D+ and show quantitative and qualitative
results in Tab. 3, Tab. 4 and Fig. 8.

Table 3: Quantitative results on Pascal3D+. Our method performs favorably to baselines and other SOTA methods.  Methods | FS@1$\uparrow$ | FS@5$\uparrow$ | FS@10$\uparrow$ | CD$\downarrow$
---|---|---|---|---
w/o $\mathcal{L}_{SSC}$ | 0.1363 | 0.5268 | 0.7307 | 0.898
w/o normal | 0.1185 | 0.4648 | 0.6712 | 0.952
w/o noise-tol | 0.1548 | 0.5875 | 0.7874 | 0.707
Ours | 0.1519 | 0.5914 | 0.7981 | 0.693
Cat3D [15] | 0.0858 | 0.3977 | 0.6155 | 1.118
SSMP [55] | 0.1014 | 0.4366 | 0.6614 | 1.000
Table 4: Additional quantitative comparison to SS3D on Pascal3D+. Shapes are aligned via brute-force search. Our method performs favorably to SS3D on Pascal3D+.  Methods | FS@1$\uparrow$ | FS@5$\uparrow$ | FS@10$\uparrow$ | CD$\downarrow$
---|---|---|---|---
SS3D [1] | 0.0533 | 0.4879 | 0.7702 | 0.696
Ours | 0.0585 | 0.5388 | 0.8507 | 0.572
Figure 8: Qualitative comparison on Pascal3D. Our method learns better global
3D structure and shape details than other baselines.

Ablation Study. We perform a similar ablation to Pix3D on Pascal3D+. The
results corroborate the findings from Pix3D; we verify the effectiveness of
our proposed techniques both quantitatively (Tab. 3) and qualitatively (Fig.
8).

SOTA Comparison. We first compare our method with Cat3D and SSMP. Due to the
instability of adversarial regularization and the lack of synthetic
pretraining, both methods cannot scale up well to more diverse real-world
scenarios and compare poorly to our method as demonstrated in Tab. 3. In Tab.
4, we further compare to SS3D which also uses synthetic pretraining. Because
SS3D cannot predict viewpoints, we evaluate reconstructed shapes via the
brute-force pose alignment for both our method and SS3D. In the quantitative
comparison, our method outperforms SS3D by a large margin; we also learn
better global structures and more accurate local details in the qualitative
examples, as shown in Fig. 8.

###  4.6 OpenImages

We perform experiments on OpenImages and show a qualitative comparison to SS3D
in Fig. 6 across various categories. As shown in the figure, our method
performs favorably to SS3D by reconstructing more accurate shapes both
globally and locally. These results verify the effectiveness and scalability
of our method.

###  4.7 Limitations

Although the results we achieve are promising, we find our method still can
not work well for categories that are often occluded, or for largely
deformable categories. We also do not handle the shape misalignment in our
semantic constraint explicitly, which can be detrimental for categories with
complex/deformable shapes. We think future exploration along these directions
would be exciting.

##  5 Conclusion

We present a novel model that reconstructs 3D object shapes over real-world
single-view images in a scalable way. Our model is driven by two key
techniques, the CLIP-based semantic constraint and the local geometric
constraint. These two techniques significantly benefit the global shape
understanding and local geometry reconstruction. They enable us to achieve
SOTA performance on three challenging real-world datasets containing various
objects.

Acknowledgement: This work was supported in part by NIH R01HD104624-01A1 and a
gift from Google.

## References

  * [1] Kalyan Vasudev Alwala, Abhinav Gupta, and Shubham Tulsiani.  Pre-train, self-train, distill: A simple recipe for supersizing 3d reconstruction.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3773–3782, 2022.
  * [2] Lucas Beyer, Alexander Hermans, and Bastian Leibe.  Biternion nets: Continuous head pose regression from discrete training labels.  In German Conference on Pattern Recognition, pages 157–168. Springer, 2015.
  * [3] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al.  Shapenet: An information-rich 3d model repository.  arXiv preprint arXiv:1512.03012, 2015.
  * [4] Christopher B Choy, Danfei Xu, JunYoung Gwak, Kevin Chen, and Silvio Savarese.  3d-r2n2: A unified approach for single and multi-view 3d object reconstruction.  In European conference on computer vision, pages 628–644. Springer, 2016.
  * [5] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  * [6] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv preprint arXiv:2010.11929, 2020.
  * [7] Ainaz Eftekhar, Alexander Sax, Jitendra Malik, and Amir Zamir.  Omnidata: A scalable pipeline for making multi-task mid-level vision datasets from 3d scans.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10786–10796, 2021.
  * [8] Matheus Gadelha, Subhransu Maji, and Rui Wang.  3d shape induction from 2d views of multiple objects.  In 2017 International Conference on 3D Vision (3DV), pages 402–411. IEEE, 2017.
  * [9] Georgia Gkioxari, Jitendra Malik, and Justin Johnson.  Mesh r-cnn.  In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
  * [10] Shubham Goel, Angjoo Kanazawa, and Jitendra Malik.  Shape and viewpoint without keypoints.  In European Conference on Computer Vision, pages 88–104. Springer, 2020.
  * [11] Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan Russell, and Mathieu Aubry.  AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation.  In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2018.
  * [12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [13] Paul Henderson and Vittorio Ferrari.  Learning single-image 3d reconstruction by generative modelling of shape, pose and shading.  International Journal of Computer Vision, pages 1–20, 2019.
  * [14] Philipp Henzler, Niloy J Mitra, and Tobias Ritschel.  Escaping plato’s cave: 3d shape from adversarial rendering.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9984–9993, 2019.
  * [15] Zixuan Huang, Stefan Stojanov, Anh Thai, Varun Jampani, and James M Rehg.  Planes vs. chairs: Category-guided 3d shape learning without any 3d cues.  In European Conference on Computer Vision, pages 727–744. Springer, 2022.
  * [16] Eldar Insafutdinov and Alexey Dosovitskiy.  Unsupervised learning of shape and pose with differentiable point clouds.  arXiv preprint arXiv:1810.09381, 2018.
  * [17] Angjoo Kanazawa, Shubham Tulsiani, Alexei A Efros, and Jitendra Malik.  Learning category-specific mesh reconstruction from image collections.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 371–386, 2018.
  * [18] Hiroharu Kato and Tatsuya Harada.  Learning view priors for single-view 3d reconstruction.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9778–9787, 2019.
  * [19] Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada.  Neural 3d mesh renderer.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3907–3916, 2018.
  * [20] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [21] Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al.  The open images dataset v4.  International Journal of Computer Vision, 128(7):1956–1981, 2020\.
  * [22] Xueting Li, Sifei Liu, Kihwan Kim, Shalini De Mello, Varun Jampani, Ming-Hsuan Yang, and Jan Kautz.  Self-supervised single-view 3d reconstruction via semantic consistency.  In European Conference on Computer Vision, pages 677–693. Springer, 2020.
  * [23] Chen-Hsuan Lin, Chen Kong, and Simon Lucey.  Learning efficient point cloud generation for dense 3d object reconstruction.  In proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  * [24] Chen-Hsuan Lin, Chaoyang Wang, and Simon Lucey.  Sdf-srn: Learning signed distance 3d object reconstruction from static images.  arXiv preprint arXiv:2010.10505, 2020.
  * [25] Shichen Liu, Tianye Li, Weikai Chen, and Hao Li.  Soft rasterizer: A differentiable renderer for image-based 3d reasoning.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7708–7717, 2019.
  * [26] William E Lorensen and Harvey E Cline.  Marching cubes: A high resolution 3d surface construction algorithm.  ACM siggraph computer graphics, 21(4):163–169, 1987.
  * [27] Ricardo Martin-Brualla, Noha Radwan, Mehdi SM Sajjadi, Jonathan T Barron, Alexey Dosovitskiy, and Daniel Duckworth.  Nerf in the wild: Neural radiance fields for unconstrained photo collections.  arXiv preprint arXiv:2008.02268, 2020.
  * [28] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger.  Occupancy networks: Learning 3d reconstruction in function space.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4460–4470, 2019.
  * [29] Tom Monnier, Matthew Fisher, Alexei A. Efros, and Mathieu Aubry.  Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency.  In ECCV, 2022.
  * [30] Siva Karthik Mustikovela, Varun Jampani, Shalini De Mello, Sifei Liu, Umar Iqbal, Carsten Rother, and Jan Kautz.  Self-supervised viewpoint learning from image collections.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3971–3981, 2020.
  * [31] KL Navaneet, Ansu Mathew, Shashank Kashyap, Wei-Chih Hung, Varun Jampani, and R Venkatesh Babu.  From image collections to point clouds with self-supervised shape and pose networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1132–1140, 2020.
  * [32] Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger.  Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3504–3515, 2020.
  * [33] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In International Conference on Machine Learning, pages 8748–8763. PMLR, 2021.
  * [34] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun.  Vision transformers for dense prediction.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12179–12188, 2021.
  * [35] René Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun.  Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer.  IEEE transactions on pattern analysis and machine intelligence, 2020\.
  * [36] Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li.  Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2304–2314, 2019.
  * [37] Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick.  Training region-based object detectors with online hard example mining.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 761–769, 2016.
  * [38] Alessandro Simoni, Stefano Pini, Roberto Vezzani, and Rita Cucchiara.  Multi-category mesh reconstruction from image collections.  arXiv preprint arXiv:2110.11256, 2021.
  * [39] Xingyuan Sun, Jiajun Wu, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B Tenenbaum, and William T Freeman.  Pix3d: Dataset and methods for single-image 3d shape modeling.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  * [40] Maxim Tatarchenko, Stephan R Richter, René Ranftl, Zhuwen Li, Vladlen Koltun, and Thomas Brox.  What do single-view 3d reconstruction networks learn?  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3405–3414, 2019.
  * [41] Anh Thai, Stefan Stojanov, Vijay Upadhya, and James M Rehg.  3d reconstruction of novel object shapes from single images.  In 2021 International Conference on 3D Vision (3DV), pages 85–95. IEEE, 2021.
  * [42] Shubham Tulsiani, Tinghui Zhou, Alexei A Efros, and Jitendra Malik.  Multi-view supervision for single-view reconstruction via differentiable ray consistency.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2626–2634, 2017.
  * [43] Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang.  Pixel2mesh: Generating 3d mesh models from single rgb images.  In Proceedings of the European conference on computer vision (ECCV), pages 52–67, 2018.
  * [44] Chao Wen, Yinda Zhang, Zhuwen Li, and Yanwei Fu.  Pixel2mesh++: Multi-view 3d mesh generation via deformation.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 1042–1051, 2019.
  * [45] Jiajun Wu, Yifan Wang, Tianfan Xue, Xingyuan Sun, Bill Freeman, and Josh Tenenbaum.  Marrnet: 3d shape reconstruction via 2.5 d sketches.  Advances in neural information processing systems, 30, 2017.
  * [46] Shangzhe Wu, Christian Rupprecht, and Andrea Vedaldi.  Unsupervised learning of probably symmetric deformable 3d objects from images in the wild.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1–10, 2020.
  * [47] Yu Xiang, Roozbeh Mottaghi, and Silvio Savarese.  Beyond pascal: A benchmark for 3d object detection in the wild.  In IEEE winter conference on applications of computer vision, pages 75–82. IEEE, 2014.
  * [48] Haozhe Xie, Hongxun Yao, Xiaoshuai Sun, Shangchen Zhou, and Shengping Zhang.  Pix2vox: Context-aware 3d reconstruction from single and multi-view images.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 2690–2698, 2019.
  * [49] Qiangeng Xu, Weiyue Wang, Duygu Ceylan, Radomir Mech, and Ulrich Neumann.  Disn: Deep implicit surface network for high-quality single-view 3d reconstruction.  arXiv preprint arXiv:1905.10711, 2019.
  * [50] Xinchen Yan, Jimei Yang, Ersin Yumer, Yijie Guo, and Honglak Lee.  Perspective transformer nets: Learning single-view 3d object reconstruction without 3d supervision.  arXiv preprint arXiv:1612.00814, 2016.
  * [51] Guandao Yang, Yin Cui, Serge Belongie, and Bharath Hariharan.  Learning single-view 3d reconstruction with limited pose supervision.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 86–101, 2018.
  * [52] Linjie Yang, Ping Luo, Chen Change Loy, and Xiaoou Tang.  A large-scale car dataset for fine-grained categorization and verification.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3973–3981, 2015.
  * [53] Yuan Yao, Nico Schertler, Enrique Rosales, Helge Rhodin, Leonid Sigal, and Alla Sheffer.  Front2back: Single view 3d shape reconstruction via front to back prediction.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 531–540, 2020.
  * [54] Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman.  Volume rendering of neural implicit surfaces.  In Thirty-Fifth Conference on Neural Information Processing Systems, 2021.
  * [55] Yufei Ye, Shubham Tulsiani, and Abhinav Gupta.  Shelf-supervised mesh prediction in the wild.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8843–8852, 2021.
  * [56] Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sattler, and Andreas Geiger.  Monosdf: Exploring monocular geometric cues for neural implicit surface reconstruction.  arXiv preprint arXiv:2206.00665, 2022.
  * [57] Junzhe Zhang, Daxuan Ren, Zhongang Cai, Chai Kiat Yeo, Bo Dai, and Chen Change Loy.  Monocular 3d object reconstruction with gan inversion.  In European Conference on Computer Vision, pages 673–689. Springer, 2022.

##  Appendix A Generalization performance

Performance on unseen categories. We train on 6 categories (car, chair,
diningtable, motorbike, train, tvmonitor) of Pascal3D+ and test on other
unseen categories (see Fig. 9 (a)). We find our model can generalize to
categories that are highly related to at least one training category (e.g.
sofa - average CD 0.571), and does not generalize as well to categories less
related to training categories (e.g. bottle - average CD 1.020).

We further quantify the relationship between generalization performance and
semantic relevance. We analyze the correlation between reconstruction error
and minimum CLIP distance from each test sample to training images. As in Fig.
10 (b), there exists a clear positive correlation (Pearson coefficient $\rho$
= 0.53) between the two variables. This verifies our model often generalizes
better to samples that are more semantically related to the seen categories.

Figure 9: (a) Reconstruction on unseen categories. (b) Inference of our
Pix3D-trained model on CO3D chairs.

Direct inference on in-the-wild data. Our method can also reconstruct faithful
shapes under reasonable domain gaps. We test our Pix3D model directly on CO3D
chair images (without fine-tuning) and find most reconstructions are
reasonable. See Fig. 9 (b) for examples.

Figure 10: (a) Histogram of viewpoint distance from query images to 5 CLIP
neighbors on Pix3D. (b) Correlation between reconstruction error and minimum
CLIP distance on unseen categories.

##  Appendix B Additional Analysis of the Model

Viewpoint robustness of CLIP embeddings. We quantitatively evaluate the
viewpoint robustness of CLIP embeddings on Pix3D chairs. In our experiments,
CLIP can find a significant number of neighbors with distinct viewpoints (see
Fig. 10 (a)). Geodesic distance is the minimal angular difference between two
rotations. The average geodesic distance from query images to CLIP neighbors
is 64°, and 67% of the query images have at least one neighbor with distinct
pose (at least 90°away).

Robustness to corrupted masks. To evaluate the performance under corrupted
masks, we replace the Pix3D masks with masks corrupted by perlin noise and
then train/test our models under different level of pixel corruption
percentages. Under 0/5/10/20/30/50% corruption level, the chamfer distance is
0.612/0.626/0.632/0.651/0.689/0.763 respectively. This experiment shows that
our model is not significantly affected by mild to moderate mask inaccuracy.

Performance with GT viewpoints. We further evaluate our model when GT
viewpoints are given during training. An image can be explained by infinite
combinations of shapes and viewpoints. When GT viewpoints are given, such
entanglement is resolved and the learning will be much easier. Our model
trained with GT viewpoints obtains average CD of 0.418 on Pix3D (vs. 0.612 w/o
GT viewpoint).

Additional discussion on retrieval methods. Comparing reconstruction methods
to retrieval methods has been one of the central topics in the area of single-
view shape reconstruction [40]. Based on the finding about CLIP’s relationship
to shape in our paper, it would be natural to consider the retrieval baseline
using CLIP. While retrieving shapes with CLIP is an interesting direction, we
would like to emphasize that it is not directly comparable to our proposed
reconstruction method. Retrieval methods require a large paired image-3D shape
database, similar to the non-scalable fully supervised 3D reconstruction
setting. In contrast, our method only requires single-view 2D images during
training, allowing it to learn to reconstruct objects from datasets like
OpenImages for which there are no paired 3D shapes. Such datasets without any
geometric annotations are our main application domain, and retrieval methods
cannot be applied to these datasets.

##  Appendix C Additional Implementation Details.

Implicit representation and rendering. The surface representation and texture
rendering follow [54, 56]. We use an implicit SDF field and convert it to
densities for volumetric rendering. The conversion from SDF to densities is
done via the Cumulative Distribution Function (CDF) of the Laplace
distribution:

$\sigma(s)=\begin{cases}\frac{1}{\beta}\cdot\frac{1}{2}\exp(\frac{s}{\beta})&\text{if
$s\leq 0$}\\\
\frac{1}{\beta}\cdot(1-\frac{1}{2}\exp(-\frac{s}{\beta}))&\text{if
$s>0$}\end{cases},$ (9)

where $s$ is the SDF. Because our focus is shape, and learning radiance is
challenging without viewpoint annotation, we represent texture as an RGB field
without any view-dependency. The surface normal rendering follows MonoSDF
[56], where the local normal vectors are estimated by the gradient of the SDF
field and aggregated via the standard volume rendering.

Uniform viewpoint prior. We use a uniform prior to regularize the viewpoint
learning, which helps to prevent the rotation estimation from degeneration.
Specifically, for each minibatch during training, we estimate the empirical
distribution of the predicted azimuth. We then minimize the Earth-Mover
Distance (EMD) between the empirical azimuth distribution and a uniform prior
within $[0\degree,360\degree]$.

##  Appendix D Additional Results on OpenImages

We show more reconstruction results of our model trained on all 53 OpenImages
[21] categories used in [55]. We further compare with SS3D [1] qualitatively,
where our model demonstrates state-of-the-art reconstruction performance. Note
the training is category-specific and the performance may be further improved
by training a joint model via distillation [1], which is parallel to our
research direction here.

Figure 11: Additional qualitative results and comparison on full OpenImages.
Figure 12: Additional qualitative results and comparison on full OpenImages.