\begin{table}[t]
\small
\centering
\setlength\tabcolsep{1.0pt}
\def1.1{1.1}
\begin{tabular}{C{4.8cm}|C{1.1cm}|C{1.1cm}}
\specialrule{.1em}{.05em}{.05em}
setting & PSNR $\uparrow$ & SSIM $\uparrow$ \\ \hline
w/o Mesh Inpainter & 17.52 & 0.7064  \\ 
w/o disentanglement & 17.53 & 0.7167  \\
w/o geo. cond. & 17.38 &  0.6634 \\
\textbf{Ours: full model} &  \textbf{17.61} & \textbf{0.7186} \\ \specialrule{.1em}{.05em}{.05em}
\end{tabular}
\vspace*{-3mm}
\caption{
{Ablation studies on AIST.}
}
\label{table:ablation} 
\end{table}
\noindent\textbf{Mesh feature.} Following NHP~\cite{kwon2021neural}, we sample per-vertex image features from $\mathbf{F}$, given SMPL~\cite{loper2015smpl} mesh vertices.
We project these 3D vertices onto the image plane and use bilinear interpolation to sample the corresponding image features. 
Then, we concatenate the per-vertex image features and the vertices' root joint (~\textit{i.e.}, pelvis)-relative depths, to distinguish the features of both occluded and occluding parts.
The concatenated feature is called the \textcolor[RGB]{98,163,59}{mesh feature} and denoted as \textcolor[RGB]{98,163,59}{$\mathbf{M} \in \mathbb{R}^{M \times (c_p+1)}$}, where $M$ denotes the number of SMPL mesh vertices. We feed these to the geometry branch.

\noindent\textbf{Pixel feature.} We sample 3D query points from a ray of a target view (rendering view), following the quadrature rule of the volume rendering discussed by Max~\cite{max1995optical}. The rays are bounded by the given mesh's 3D bounding box, following NB~\cite{peng2021neural} and NHP~\cite{kwon2021neural}. 
The 3D query point $\mathbf{x}\in \mathbb{R}^3$ is equipped with appearance information from its projection on the image plane. We coin this information the \textcolor[RGB]{252,99,159}{pixel feature $\mathbf{p} \in \mathbb{R}^{c_p}$}.
The pixel feature $\mathbf{p}$ is fed to both geometry and texture branches with $\mathbf{x}$'s \textcolor[RGB]{54,93,183}{root joint-relative depth $\mathrm{z}$}.
The root joint-relative depth is defined in the input view's camera-centered coordinate system.

\noindent\textbf{Sparse 3D convolution.} 
We work with a sparse 3D volume, in which lie the sparse features of $\mathbf{M}$.
This sparse 3D volume is fed to a sparse 3D convolutional neural network (CNN)~\cite{graham2018spconv} that 
extracts multi-scale feature volumes, denoted mesh volumes.
From each mesh volume, we sample the feature corresponding to $\mathbf{x}$ based on its 3D camera coordinates using trilinear interpolation to get the \textcolor[RGB]{252,22,39}{voxel feature $\mathbf{v} \in \mathbb{R}^{c_v}$}, where
$c_v=192$ is the feature's channel dimension.
The voxel feature $\mathbf{v}$ is used only for the geometry estimation; therefore, it is a \emph{geometry-dedicated feature}.

\noindent\textbf{Comparison using estimated 3D meshes.}
We also compare MonoNHR with NHP~\cite{kwon2021neural} on ZJU-MoCap~\cite{peng2021neural} using estimated body meshes of SPIN~\cite{kolotouros2019learning} in Table~\ref{table:sota_mesh_estimate}.
SPIN is a monocular method that regresses SMPL~\cite{loper2015smpl} parameters to produce a body mesh.
MonoNHR highly outperforms NHP, which also uses a body mesh at test time, considering that PSNR is in a log scale. 
Figure~\ref{fig:estimated_sota} further validates MonoNHR's robustness on  monocular images.
MonoNHR preserves the subject's 3D shape (\textit{e.g.}, loose clothes around the torso of the right-hand subject) while NHP relatively shrinks the subject's overall shape (\textit{e.g.}, the arms of the right-hand subject).
Also, the results of MonoNHR show more realistic textures in novel views than NHP (\textit{e.g.}, the shirt of the left-hand subject).

\noindent\textbf{Comparison with non-NeRF methods.}
Figure~\ref{fig:sota_cape} shows that MonoNHR produces much better qualitative results than PIFu~\cite{saito2019pifu} and PaMIR~\cite{zheng2021pamir} on ZJU-Mocap.
Note that it is difficult to measure their PSNR/SSIM due to the pixel-level misalignment between their rendered images and the GT ones.
Table~\ref{table:sota_cape} shows that ours achieves much better CD on CAPE~\cite{ma2020learning}.
The results of PIFu and PaMIR are obtained by running their officially released codes and pre-trained weights.
We tested with MonoNHR trained on AIST.

\subsection{Ablation studies}
\noindent\textbf{Disentanglement of geometry and texture.}
To show the benefit of the proposed disentanglement, we design a model where voxel features $\mathbf{v}$ are fed to both geometry and texture branches.
Results confirm that disentangling geometry and texture improves predictions both qualitatively and quantitatively (Table~\ref{table:ablation}).
As shown in Figure~\ref{fig:ablation_disentangle}, noise on the invisible surface's texture is removed, and the overall visual quality is enhanced.

\noindent\textbf{Mesh Inpainter.}
Using the Mesh Inpainter improves results both qualitatively (Fig.~\ref{fig:ablation_mesh_inpainter}) and quantitatively (Table~\ref{table:ablation}).
The critical challenge of the monocular setting is inferring the geometry and texture of invisible areas.
To this end, the backbone should produce proper features for unseen parts based on the visible ones, which amounts to learning human priors, such as symmetry of human parts and color similarity between surfaces belonging to the same part.
While these could be learned by the rendering loss at the end of the network, we tried to facilitate it by giving a more direct signal to the backbone using mesh inpainting.

\noindent\textbf{Geometry-conditioned texture estimation.}
Conditioning on geometry when estimating texture is essential (Fig.~\ref{fig:ablation_geo_condition} and Table~\ref{table:ablation}).
Indeed, geometry provides information on whether a 3D query point is occupied or not, and such occupancy information is important to produce accurate textures at correct locations.
{"value": "17.52", "type": "Result", "training data/set": "AIST", "test data/set": "AIST", "task": ["human rendering", "free-viewpoint renderings"], "metric": ["PSNR", "peak signal-to-noise ratio"], "experimental settings": {"xx": "yy"}, "model": "MonoNHR", "model settings": {"setting": "w/o Mesh Inpainter"}} {"value": "0.7064", "type": "Result", "training data/set": "AIST", "test data/set": "AIST", "task": ["human rendering", "free-viewpoint renderings"], "metric": ["SSIM", "structural similarity index"], "experimental settings": {"xx": "yy"}, "model": "MonoNHR", "model settings": {"setting": "w/o Mesh Inpainter"}} {"value": "17.53", "type": "Result", "training data/set": "AIST", "test data/set": "AIST", "task": ["human rendering", "free-viewpoint renderings"], "metric": ["PSNR", "peak signal-to-noise ratio"], "experimental settings": {"xx": "yy"}, "model": "MonoNHR", "model settings": {"setting": "w/o disentanglement"}} {"value": "0.7167", "type": "Result", "training data/set": "AIST", "test data/set": "AIST", "task": ["human rendering", "free-viewpoint renderings"], "metric": ["SSIM", "structural similarity index"], "experimental settings": {"xx": "yy"}, "model": "MonoNHR", "model settings": {"setting": "w/o disentanglement"}} {"value": "17.38", "type": "Result", "training data/set": "AIST", "test data/set": "AIST", "task": ["human rendering", "free-viewpoint renderings"], "metric": ["PSNR", "peak signal-to-noise ratio"], "experimental settings": {"xx": "yy"}, "model": "MonoNHR", "model settings": {"setting": "w/o geo. cond."}} {"value": "0.6634", "type": "Result", "training data/set": "AIST", "test data/set": "AIST", "task": ["human rendering", "free-viewpoint renderings"], "metric": ["SSIM", "structural similarity index"], "experimental settings": {"xx": "yy"}, "model": "MonoNHR", "model settings": {"setting": "w/o geo. cond."}} {"value": "17.61", "type": "Result", "training data/set": "AIST", "test data/set": "AIST", "task": ["human rendering", "free-viewpoint renderings"], "metric": ["PSNR", "peak signal-to-noise ratio"], "experimental settings": {"xx": "yy"}, "model": "MonoNHR", "model settings": {"setting": "full model"}} {"value": "0.7186", "type": "Result", "training data/set": "AIST", "test data/set": "AIST", "task": ["human rendering", "free-viewpoint renderings"], "metric": ["SSIM", "structural similarity index"], "experimental settings": {"xx": "yy"}, "model": "MonoNHR", "model settings": {"setting": "full model"}}