Patients with asymmetric glaucomatous optic neuropathy (GON) present a relative afferent pupillary defect (RAPD) in the eye with more advanced damage. Although useful, pupillometric RAPD quantification is not widely used as it is not portable. Whether asymmetry of the peripapillary capillary perfusion density (CPD) detected using optical coherence tomography angiography correlates with the severity of RAPD remains unknown. This study assessed RAPD in 81 patients with GON using Hitomiru, a novel hand-held infrared binocular pupillometer. The correlation and ability to detect clinical RAPD based on the swinging flash light test of two independent RAPD parameters (the maximum pupil constriction ratio and the constriction maintenance capacity ratio) were assessed. The coefficient of determination (R2) was calculated between each of the two RAPD parameters and asymmetry of the circumpapillary retinal nerve fiber layer thickness (cpRNFLT), ganglion cell layer/inner plexiform layer thickness (GCL/IPLT), and CPD. The two RAPD parameters showed a correlation coefficient of 0.86 and areas under the receiver operating characteristic (ROC) curve of 0.85–0.88, with R2being 0.63–0.67 for the visual field, 0.35–0.45 for cpRNFLT, 0.45–0.49 for GCL/IPLT, and 0.53–0.59 for CPD asymmetry. Hitomiru has high discriminatory performance in detecting RAPD in patients with asymmetric GON. CPD asymmetry may better correlate with RAPD than cpRNFLT and GCL/IPLT asymmetry.