\begin{table}[t]
\small
\centering
\setlength\tabcolsep{1.0pt}
\def1.1{1.1}
\begin{tabular}{C{3.0cm}|C{1.1cm}|C{1.1cm}}
\specialrule{.1em}{.05em}{.05em}
method & PSNR $\uparrow$ & SSIM $\uparrow$ \\ \hline
NHP~\cite{kwon2021neural} & 17.472 & 0.7167\\ 
\textbf{MonoNHR (Ours)} &  \textbf{19.750} & \textbf{0.7463}\\
\specialrule{.1em}{.05em}{.05em}
\end{tabular}
\caption{
{Comparison with NHP~\cite{kwon2021neural} on HUMBI.} 
}
\label{table:nhp_humbi}
\end{table}
\noindent\textbf{Mesh feature.} Following NHP~\cite{kwon2021neural}, we sample per-vertex image features from $\mathbf{F}$, given SMPL~\cite{loper2015smpl} mesh vertices.
We project these 3D vertices onto the image plane and use bilinear interpolation to sample the corresponding image features. 
Then, we concatenate the per-vertex image features and the vertices' root joint (~\textit{i.e.}, pelvis)-relative depths, to distinguish the features of both occluded and occluding parts.
The concatenated feature is called the \textcolor[RGB]{98,163,59}{mesh feature} and denoted as \textcolor[RGB]{98,163,59}{$\mathbf{M} \in \mathbb{R}^{M \times (c_p+1)}$}, where $M$ denotes the number of SMPL mesh vertices. We feed these to the geometry branch.

\section{Experiments}
\subsection{Datasets}
\noindent\textbf{Datasets.}
We train and test MonoNHR on ZJU-MoCap~\cite{peng2021neural}, AIST~\cite{li2021learn,tsuchida2019aist}, CAPE~\cite{ma2020learning}, and HUMBI~\cite{yu2018humbi}.
For ZJU-MoCap, we split 10 videos of 10 subjects (1 video per 1 subject) to 7 training videos and 3 testing videos.
For AIST, we follow the official training and testing splits.
The training set has 834 videos of 20 dancing subjects, and the testing set has 374 videos of 10 dancing subjects.
Please note that the testing splits of all datasets contain poses and identities never seen during training.
CAPE is used to compare 3D geometry with non-NeRF methods as the above two datasets do not provide GT 3D geometry.
We use the above datasets for numerical studies, for more meaningful comparisons with prior work, but we also train on HUMBI for qualitative evaluation, for its larger size and diversity (see section~\ref{subsec:qualitative_results}). More details about data settings can be found in the supplementary material.

\noindent\textbf{Comparison using estimated 3D meshes.}
We also compare MonoNHR with NHP~\cite{kwon2021neural} on ZJU-MoCap~\cite{peng2021neural} using estimated body meshes of SPIN~\cite{kolotouros2019learning} in Table~\ref{table:sota_mesh_estimate}.
SPIN is a monocular method that regresses SMPL~\cite{loper2015smpl} parameters to produce a body mesh.
MonoNHR highly outperforms NHP, which also uses a body mesh at test time, considering that PSNR is in a log scale. 
Figure~\ref{fig:estimated_sota} further validates MonoNHR's robustness on  monocular images.
MonoNHR preserves the subject's 3D shape (\textit{e.g.}, loose clothes around the torso of the right-hand subject) while NHP relatively shrinks the subject's overall shape (\textit{e.g.}, the arms of the right-hand subject).
Also, the results of MonoNHR show more realistic textures in novel views than NHP (\textit{e.g.}, the shirt of the left-hand subject).

\noindent\textbf{Comparison with non-NeRF methods.}
Figure~\ref{fig:sota_cape} shows that MonoNHR produces much better qualitative results than PIFu~\cite{saito2019pifu} and PaMIR~\cite{zheng2021pamir} on ZJU-Mocap.
Note that it is difficult to measure their PSNR/SSIM due to the pixel-level misalignment between their rendered images and the GT ones.
Table~\ref{table:sota_cape} shows that ours achieves much better CD on CAPE~\cite{ma2020learning}.
The results of PIFu and PaMIR are obtained by running their officially released codes and pre-trained weights.
We tested with MonoNHR trained on AIST.

\subsection{Qualitative results} \label{subsec:qualitative_results}
The HUMBI dataset~\cite{yu2020humbi} has a high diversity of human subjects in terms of body shape, age, ethnicity, clothing and accessories. We train on this data in order to evaluate the robustness and generalization capabilities of our method.
Figures~\ref{fig:banner} and \ref{fig:humbi_inpaint_comparison} show some results on test subjects unseen during training. In the latter, we show the benefits of using neural rendering by comparing with surface renderings of the inpainted SMPL mesh. Despite the accuracy of the SMPL annotations on this dataset, we can clearly see the inherent limits of texture based approaches. They typically fail in regions where the true surface largely differs from the template shape which is often the case \eg with hair or wide cloths. In contrast, even though our approach makes use of the SMPL vertices, we exhibit good robustness to these cases.

\section{Qualitative results}
\label{supp:quality}
We provide a video\footnote{\url{https://youtu.be/9-hfGf7dRw4}} showing more qualitative results and comparisons on the ZJU-MoCap dataset~\cite{peng2021neural}, as well as the HUMBI dataset~\cite{yu2020humbi}. Below are more details and comments regarding the video.
\subsection{Results on ZJU-MoCap}
We show video results on ZJU-MoCap, both on dynamic scenes (where the input frame changes for each rendered image) and static scenes (where a video is generated given a single input image). Comparisons with NHP~\cite{kwon2021neural} show that our method is quite stable with respect to time, and does not suffer from the jittering effect present in the NHP results.
Other results show that MonoNHR can better render loose clothing, suggesting that it is able to correctly infer volumetric information around the underlying body-model, whereas NHP renderings tend to stick to the body-model.
Finally, in some cases, NHP has trouble generalizing to a new subject given only a single image. 

\subsection{Results on HUMBI} \label{supp:humbi}
In order to challenge the generalization capabilities of MonoNHR, we also train and test it on the HUMBI dataset~\cite{yu2020humbi}, which has a high diversity of human subjects in terms of body shape, age, race, clothing and accessories. We train our method on 9822 frames, taken from 334 subjects, with various poses. Qualitative results on diverse test subjects and poses are included in the attached video. They demonstrate that our method generalizes well to new people, unseen during training.
Also, some interesting effects can be noted in the attached video. 
For example, our method learns to correctly reconstruct long hair in the back, given only a single frontal view of a person.

We retrain NHP on HUMBI for an additional comparison. We use the same protocol as MonoNHR (see section \ref{supp:humbi}). Implementation details are given in section \ref{supp:nhp}.
Table \ref{table:nhp_humbi} shows the average numerical results on 13 test subjects, unseen during training. (For each subject, we test on 5 views of one frame).
Figure \ref{fig:nhp_humbi} shows some qualitative comparisons on some of those test subjects.
Our method produces more realistic and visually pleasing renderings. However, like NHP, it still suffers from artifacts for rendering the opposite surface of an observed view, as can be seen in the second row. These come from the difficulty of extracting proper features for novel views only from observed image features.
{"value": "17.472", "type": "Result", "training data/set": "HUMBI", "test data/set": "HUMBI", "task": ["human rendering", "free-viewpoint renderings"], "metric": ["PSNR", "peak signal-to-noise ratio"], "experimental settings": {"xx": "yy"}, "model": "NHP", "model settings": {"xx": "yy"}} {"value": "0.7167", "type": "Result", "training data/set": "HUMBI", "test data/set": "HUMBI", "task": ["human rendering", "free-viewpoint renderings"], "metric": ["SSIM", "structural similarity index"], "experimental settings": {"xx": "yy"}, "model": "NHP", "model settings": {"xx": "yy"}} {"value": "19.750", "type": "Result", "training data/set": "HUMBI", "test data/set": "HUMBI", "task": ["human rendering", "free-viewpoint renderings"], "metric": ["PSNR", "peak signal-to-noise ratio"], "experimental settings": {"xx": "yy"}, "model": "MonoNHR", "model settings": {"xx": "yy"}} {"value": "0.7463", "type": "Result", "training data/set": "HUMBI", "test data/set": "HUMBI", "task": ["human rendering", "free-viewpoint renderings"], "metric": ["SSIM", "structural similarity index"], "experimental settings": {"xx": "yy"}, "model": "MonoNHR", "model settings": {"xx": "yy"}}