Comprehensive Benchmarking of Binary Neural Networks on NVM Crossbar
  Architectures
Non-volatile memory (NVM) crossbars have been identified as a promising
technology, for accelerating important machine learning operations, with
matrix-vector multiplication being a key example. Binary neural networks (BNNs)
are especially well-suited for use with NVM crossbars due to their use of a
low-bitwidth representation for both activations and weights. However, the
aggressive quantization of BNNs can result in suboptimal accuracy, and the
analog effects of NVM crossbars can further degrade the accuracy during
inference.
  This paper presents a comprehensive study that benchmarks BNNs trained and
validated on ImageNet and deployed on NeuroSim, a simulator for
NVM-crossbar-based PIM architecture. Our study analyzes the impact of various
parameters, such as input precision and ADC resolution, on both the accuracy of
the inference and the hardware performance metrics. We have found that an ADC
resolution of 8-bit with an input precision of 4-bit achieves near-optimal
accuracy compared to the original BNNs. In addition, we have identified
bottleneck components in the PIM architecture that affect area, latency, and
energy consumption, and we demonstrate the impact that different BNN layers
have on hardware performance.
Ruirong Huang, Zichao Yue, Caroline Huang, Janarbek Matai, Zhiru Zhang
# Comprehensive Benchmarking of Binary Neural Networks on NVM Crossbar Architectures

###### Abstract

Non-volatile memory (NVM) crossbars have been identified as a promising technology, for accelerating important machine learning operations, with matrix-vector multiplication being a key example. Binary neural networks (BNNs) are especially well-suited for use with NVM crossbars due to their use of a low-bitwidth representation for both activations and weights. However, the aggressive quantization of BNNs can result in suboptimal accuracy, and the analog effects of NVM crossbars can further degrade the accuracy during inference.

This paper presents a comprehensive study that benchmarks BNNs trained and validated on ImageNet and deployed on NeuroSim, a simulator for NVM-crossbars-based PIM architecture. Our study analyzes the impact of various parameters, such as input precision and ADC resolution, on both the accuracy of the inference and the hardware performance metrics. We have found that an ADC resolution of 8-bit with an input precision of 4-bit achieves near-optimal accuracy compared to the original BNNs. In addition, we have identified bottleneck components in the PIM architecture that affect area, latency, and energy consumption, and we demonstrate the impact that different BNN layers have on hardware performance.

 Deep Learning, Binary Neural Network, Non-Volatile Memory, Processing-In-Memory

## I Introduction

Non-volatile memory (NVM) crossbars have become popular as a type of analog computing substrate used in processing-in-memory (PIM) architectures [1, 2, 3, 4]. These crossbars consist of an array of NVM cells and perform matrix-vector multiplication (MxV) in the analog domain using Ohm's law and Kirchhoff's current law. By processing data in-situ, the need for data movement is reduced, resulting in increased performance and energy efficiency. This approach is gaining traction in hardware acceleration for machine learning, where MxV is a critical computational kernel.

However, supporting high-precision neural networks in NVM crossbars can be challenging because of the need for digital-to-analog converters (DAC) and analog-to-digital converters (ADC) to convert signals from the digital domain to the analog domain for processing and vice versa. High-precision neural networks require high-resolution DAC and ADC, which consume more energy and can offset the benefits of in-situ computing. As a result, researchers often resort to quantization techniques to reduce precision while maintaining acceptable accuracy levels, making quantized neural networks a more viable option for NVM crossbars [5, 6, 7].

Binary neural networks (BNNs) are a type of quantized neural network that uses a single bit to represent weights and activations [8, 9, 10, 11, 12, 13]. The extreme quantization of BNNs can lower both computational complexity and storage requirements. When deployed in NVM crossbars, BNNs require only a simple inverter for input activation, and the shift-and-add circuit commonly used for weight bit-slicing can be eliminated, leading to greater energy efficiency. This synergy between BNNs and NVM crossbars creates a promising scenario for deploying many emerging applications at the edge settings with strict resource and latency constraints. The synergy between BNNs and NVM crossbars creates a promising scenario for deploying a variety of emerging ML workloads at the edge with strict resource and latency constraints.

While deploying BNNs in NVM crossbars holds potential benefits, it is crucial to pay close attention to the possible negative effects. BNNs typically have lower inference accuracy due to their aggressive quantization nature compared to high-precision counterparts. The analog effects of NVM crossbars, such as insufficient ADC resolution and memory non-idealities, can further decrease the inference accuracy. Additionally, the configuration of BNNs, including model structure and input precision, can significantly impact the hardware performance of NVM crossbars. Improper choices of the model configuration can offset the advantages of energy efficiency gained by using the NVM crossbar.

Several simulators have been proposed for NVM-crossbar-based neural network accelerators [14, 15, 16, 17, 18, 19] to facilitate fast design space exploration and performance evaluation. These simulators typically encompass different levels of hardware modeling, including memory, circuit, and chip, with varying degrees of fidelity. For instance, NeuroSim [19] provides detailed circuit-level modeling, while MNSIM [18] offers more flexible chip-level exploration options. Previous studies have utilized these simulators to explore the potential of NVM crossbars for BNN inference, primarily focusing on simpler BNNs [20, 21, 12] and small datasets such as MNIST [21, 22] and CIFAR-10 [23, 24, 25, 12]. However, a comprehensive benchmarking study is still lacking to investigate the effectiveness of NVM crossbars on more complex BNN models using realistic datasets. Such a benchmark could provide valuable insights into the performance bottleneck of NVM crossbar hardware components and which BNN features are best suited for use in NVM crossbars.

In this paper, we use the open-source simulator NeuroSim [19] to conduct a comprehensive benchmark of modern BNNs
implemented on NVM-crossbar-based PIM architecture. The contributions of this paper are as follows.

* This study provides the first comprehensive benchmarking of three realistic BNN models (BAlexNet, BResNet, and BDenseNet) on an NVM-crossbar-based PIM architecture, trained and validated on the ImageNet dataset. We analyze the impact of various parameters, including input precision and ADC resolution, on both inference accuracy and chip performance, such as chip area, latency, energy consumption, and throughput.
* We analyze the relationship between inference accuracy, input precision, and ADC resolution, and demonstrate that increasing the ADC resolution and input precision can generally result in higher accuracy but lower throughput. Our study shows that an ADC resolution of 8-bit with an input precision of 4-bit offers a negligible accuracy loss compared to the same BNN model evaluated on a GPU with float32 first-layer activation, while achieving a better energy efficiency trade-off.
* We identify the hardware components that bottleneck the performance of the NVM crossbar architecture. Specifically, our work demonstrates that the area of the ADC grows exponentially with increasing resolution, becoming the dominant part of the chip in terms of area. Additionally, we find that buffer latency and interconnection dynamic energy are major contributors to latency and energy consumption, respectively.
* We demonstrate how different model structures impact inference accuracy and hardware performance. Our findings reveal that for deeper models, the pipelined architecture of the NVM accelerator can significantly reduce latency, but large fully-connected layers become the bottleneck of chip performance, especially in terms of latency.

## II Background

### _Binary Neural Network_

Binary Neural Network [8, 9, 10, 11, 12, 13] is a type of neural network with binary weights and activations. In a BNN, each weight and activation value is either +1 or -1, which takes only 1 bit of memory, rather than a floating point or integer value, which takes 32 or 16 bits.

The use of binary weights and activations in BNNs offers several advantages over traditional neural networks, including reduced memory requirements, faster processing speed, and improved energy efficiency. Additionally, BNNs have been shown to be more robust to noisy inputs and adversarial attacks compared to traditional neural networks. This is due to the reduced sensitivity of binary values to small perturbations in the input.

However, BNNs also have some limitations compared to traditional neural networks. The binary weights and activations result in a loss of expressiveness and accuracy compared to their higher bit-precision counterparts. Additionally, training BNNs can be challenging, as the non-differentiable binary activation function can make it difficult to perform back-propagation and update the weights.

### _Processing-in-Memory_

Processing-in-Memory (PIM) architecture is a computer architecture that endows the memory module with computing capability [26, 27, 28]. One of the main benefits of PIM is reduced data movement and communication overhead. In traditional architectures, data must be transferred from memory to the processor for processing, which significantly increases the energy consumption and latency. PIM eliminates this transfer, as the data is processed directly in the memory, hence improving performance and energy efficiency. In this paper, we focus on an emerging PIM device known as the NVM crossbar.

#### Ii-B1 Embedded Non-volatile Memory

Non-volatile memory (NVM) is a type of memory that can retain data even when power is not supplied. Several emerging NVM technologies, such as resistive random access memory (RRAM), phase change random access memory (PCM), and magnetoresistive random access memory (MRAM), offer comparable or shorter latency compared to dynamic random access memory (DRAM), along with higher density than DRAM and static random access memory (SRAM) [29]. Embedded non-volatile memory (eNVM) specifically refers to NVM integrated into a system-on-chip (SoC) or an application-specific integrated circuit (ASIC) [30, 31, 32]. In addition to the advantages of NVM, eNVM can be manufactured using standard CMOS processes, making it relatively low-cost and easy to integrate into existing designs. These benefits make eNVM-based crossbar an attractive option for ML workloads. In this paper, we focus on RRAM, which uses material that can be switched between two or multiple resistance states to store one or more bits of information [33, 4, 32]. We use the 22nm RRAM model provided in NeuroSim for our benchmark.

#### Ii-B2 NVM Crossbar

NVM crossbar [1, 2, 3, 4] is a type of PIM architecture that enables parallel processing and data storage by integrating memory and computation functions within a single device. In the crossbar architecture, the memory cells are arranged in a two-dimensional array of rows and columns, and each cell is connected to both a _wordline_ and a _bitline_. One of the main advantages of a crossbar architecture is its ability to perform massively parallel operations, which can greatly accelerate computation and reduce power consumption. In addition, the crossbar can be implemented using a variety of NVM technologies, such as RRAM, PCM or MRAM, which can provide both high-density and low-power storage.

Matrix-vector multiplication is usually the dominant computation in ML workloads, which can be done in parallel through crossbar devices [21, 34, 2] with each crosspoint storing one weight. With the input activations applied as voltage to horizontal lines, the vector-matrix multiplication is automatically down through the principles of Ohm's law and Kirchhoff's current law. By sensing the amplitude of the output current at each vertical line, we can read out the vector-matrix multiplication results in parallel.

#### Ii-B3 NeuroSim

NeuroSim is a simulation platform designed for modeling and analyzing the performance of deep neural network (DNN) accelerators adopting NVM-crossbar-based
architectures [20, 22, 23]. NeuroSim provides hardware models that cover different levels of abstraction, including memory, circuit, and chip architecture. Compared to other open-source simulators such as CrossSim [14], PytorX [15] and MNSIM [18], NeuroSim offers more detailed circuit-level models, allowing for accurate estimation of inference accuracy and hardware performance, including area, latency, and energy consumption. However, it is important to note that one limitation of NeuroSim is that the accelerator floorplans generated for different neural networks are model-specific.

## III Benchmark Design

In this section, we present our benchmark design, which involves three BNN models: BAlexnet, BResnet18, BDensenet28, each with two focuses: inference accuracy and hardware performance. First, we introduce the models used in our experiments, and then we detail our benchmark design and methodology.

### _Models_

NeuroSim provides a PyTorch wrapper for simulating the inference accuracy of different neural network models. We build BAlexNet, BResNet18, and BDenseNet28 in PyTorch referring to their TensorFlow-version from Larq Zoo [35].

#### Iii-A1 AlexNet

AlexNet [36] is a convolutional neural network (CNN) architecture consisting of five convolution layers and three large fully-connected layers. The convolution layers employ a combination of filters of varying sizes and strides, which are then followed by max-pooling layers. The fully-connected layers are succeeded by a softmax layer that produces the class probabilities.

#### Iii-A2 ResNet

ResNet, [37] short for "Residual Network", is a CNN architecture that introduces residual connections, which allow information to be passed directly from one layer to another. This is done by adding the output of a previous layer to the input of a subsequent layer, allowing the network to learn with a much deeper size and produce better accuracy. Various versions of ResNet exist with differing numbers of layers. In our benchmark, we employ ResNet18.

#### Iii-A3 DenseNet

DenseNet, [38] short for "Densely Connected CNN", is a deep neural network architecture that introduces densely connected blocks, which are made up of several convolution layers with a fixed number of filters. Each layer receives the feature maps of all preceding layers as inputs, concatenated together along the depth dimension. This dense connectivity pattern allows the network to reuse features learned by earlier layers, which results in better parameter efficiency and gradient flow. In our benchmark, we utilize DenseNet28.

### _Inference Accuracy_

Accuracy is one of the biggest concerns in BNNs, while model inference is the most common use case for edge machine learning devices. In order to get realistic results of inference accuracy for BNN models on large datasets, we first train all three models on ImageNet using binary weights for all layers and binary activations for all but the first layer. Next, we evaluate the trained model using the NeuroSim simulator by replacing key functions such as nn.conv2d with APIs provided by NeuroSim that account for hardware effects. However, since NeuroSim is not specifically designed for BNNs, it employs a high-precision floating-point input for the first layer, with bit-serialization modules used to process activations in the subsequent layers. To better simulate BNNs, we introduce a bit-serialization module to process the input of the first layer and eliminate this module from all subsequent layers, given that the activations of these layers are already binarized. We also employ dynamic quantization of input data. To further investigate, we focus on two parameters with the most significant impact on accuracy: first-layer input precision and the resolution of ADCs.

### _Hardware Performance_

To quantify the performance of BNNs deployed in NVM crossbar architectures, we first provide layer-level model structures for NeuroSim, since it's a model-specific simulator. Each layer includes the dimension of its input activation and weight, as well as parameters for any potential subsequent pooling and activation module (i.e. relu). Furthermore, we collect the intermediate activation and weight data for each layer from the trained model, as this information is crucial for NeuroSim to generate more precise estimates based on the distribution of each layer's input activation and weight. NeuroSim offers different modes catering to distinct design considerations. In our benchmark, we select the _XnorParallel_ mode, which employs two memory cells to represent a weight of either +1 or -1, making it suitable for BNNs. Moreover, we activate the pipeline mode in NeuroSim to leverage the pipelined architecture of the chip, thereby achieving enhanced throughput. To demonstrate the impact of ADC resolution on both accuracy and hardware performance, we perform tests on each model using various ADC resolutions and collect their hardware performance data, including chip area, latency, energy consumption, throughput, and efficiency.

## IV Experiments

This section provides a detailed explanation of our experiment parameters and results, divided into four parts: inference accuracy, area, latency, and energy, each with four subsections corresponding to the three BNN models. Additionally, there is a Put-It-All-Together section that compares the results of the three models. Finally, we analyze the throughput and efficiency of the three models.

### _Inference Accuracy_

#### Iv-A1 AlexNet

Table I presents the inference accuracy for Binary AlexNet across different ADC resolutions (3-8) and first-layer input precisions (1-8). The same data is used to generate Figure 1 and Figure 2, which visually depict the trend of accuracy change. In Figure 1, the accuracy for input precision of 1 increases with ADC resolution but remains low at the highest precision (7.55% for ADC resolution=8).
Accuracy improves significantly with an input precision of 2, reaching a maximum of 30.71%. With an input precision of 3, accuracy reaches an asymptote, and increasing input precision beyond 3 has little effect on accuracy.

Figure 2 also shows that accuracy monotonically increases with ADC resolution, although this trend stops when ADC resolution reaches 7. Overall, input precision of 4-bit with ADC resolution of 8-bit provides an acceptable accuracy of 35.60%, which is only 0.8% lower than the maximum accuracy of 35.9% when both parameters are set to 8, and is only 1.4% lower than the accuracy of 36.11% achieved on GPU with float32 first-layer precision.

#### Iv-C2 ResNet

Table II lists the inference accuracy according to different ADC resolutions (3-8) and first-layer input precisions (1-8) for Binary ResNet. The same data is used to draw Figure 3 and Figure 4. Similar to the results of AlexNet, in Figure 3, the accuracy is still quite low when input precision is 1. It gets a great improvement when input precision increases to 2, where the max accuracy reaches 39.72%.

\begin{table}
\begin{tabular}{c|c c c c c c} \hline \hline \multirow{2}{*}{
\begin{tabular}{c} **Input** \\ **Precision** \\ \end{tabular} } & \multicolumn{5}{c}{**ADC Resolution**} \\
2 & 0.60 & 15.23 & 18.82 & 38.60 & 36.08 & 39.72 \\
3 & 0.55 & 22.35 & 24.17 & 45.86 & 48.36 & 51.81 \\
4 & 0.52 & 25.08 & 25.67 & 46.79 & 51.08 & 53.61 \\
5 & 0.70 & 26.04 & 26.04 & 46.89 & 51.60 & 53.98 \\
6 & 0.79 & 26.32 & 26.61 & 47.19 & 51.93 & 53.81 \\
7 & 0.65 & 26.51 & 26.68 & 47.08 & 52.02 & 53.82 \\
8 & 0.77 & 26.58 & 26.42 & 47.00 & 51.88 & 53.84 \\ \hline \hline \end{tabular}
\end{table} TABLE II: **Inference accuracy for Binary ResNet with different input precisions and ADC resolutions — The inference accuracy on GPU with float32 first-layer activation is 54.02%.**

\begin{table}
\begin{tabular}{c|c c c c c c} \hline \hline \multirow{2}{*}{
\begin{tabular}{c} **Input** \\ **Precision** \\ \end{tabular} } & \multicolumn{5}{c}{**ADC Resolution**} \\
2 & 0.17 & 1.43 & 3.64 & 6.07 & 7.67 & 7.55 \\
3 & 0.21 & 4.35 & 13.81 & 29.30 & 34.84 & 35.60 \\
4 & 0.16 & 4.95 & 15.51 & 30.76 & 35.57 & 35.76 \\
5 & 0.15 & 5.18 & 15.96 & 31.06 & 35.52 & 36.11 \\
6 & 0.18 & 5.24 & 16.00 & 30.99 & 35.54 & 35.84 \\
7 & 0.29 & 5.31 & 16.18 & 31.28 & 35.43 & 35.90 \\
8 & 0.31 & 5.31 & 16.31 & 31.22 & 35.56 & 35.90 \\ \hline \hline \end{tabular}
\end{table} TABLE I: **Inference accuracy for Binary AlexNet with different input precisions and ADC resolutions — The inference accuracy on GPU with float32 first-layer activation is 36.11%.**

Fig. 1: **Binary AlexNet inference accuracy vs. ADC resolution — Each line corresponds to a different first-layer input precision.**

Fig. 3: **Binary ResNet inference accuracy vs. ADC resolution — Each line corresponds to a different first-layer input precision.**

Fig. 2: **Binary AlexNet inference accuracy vs. first layer input precision — Each line corresponds to a different ADC resolution.**
Overall, input precision of 4-bit with ADC resolution of 8-bit provides an acceptable accuracy of 53.61%, which is only 0.4% lower than the maximum accuracy of 53.84% when both parameters are set to 8, and is only 0.7% lower than the accuracy of 54.02% achieved on GPU with float32 first-layer precision.

#### Iv-A3 DenseNet

Table III presents the inference accuracy for Binary DenseNet across different ADC resolutions (3-8) and first-layer input precisions (1-8). Similar to the result of ResNet, the accuracy when input precision is 1 is still quite low. It gets great improvement when input precision increases to 2, where the max accuracy reaches 40.85%.

Overall, input precision of 4-bit with ADC resolution of 8-bit provides an acceptable accuracy of 55.88%, which is only 0.08% lower than the maximum accuracy of 55.93% when both parameters are set to 8, and is only 0.5% lower than the accuracy of 56.15% achieved on GPU with float32 first-layer precision.

#### Iv-A4 Put It All Together

The maximum accuracy achieved by the three models exhibits a monotonic increase: 35.90% for AlexNet, 53.84% for ResNet, and 55.93% for DenseNet. Although the values differ, the peak accuracy for each model is attained when the input precision and ADC resolution are set to their maximum values (8-bit). Overall, input precision of 4-bit with ADC resolution of 8-bit provides negligible accuracy loss compared to the same BNN model evaluated on GPU with float32 first-layer activation.

### _Area_

#### Iv-B1 AlexNet

Figure 5 displays the chip area and its components as a function of ADC resolution for Binary AlexNet. It is evident that the total area of the ADC increases at the fastest rate as the ADC resolution increases, and this increase is the primary reason for the growth in the chip area. This is reasonable because higher ADC resolution necessitates a larger ADC circuit area. The total accumulation circuits on the chip, although having the lowest value among these components, increased by 76% from 1.05 \(mm^{2}\) to 1.85 \(mm^{2}\). The other peripheries, which occupy most of the chip area when the ADC resolution is low, have only a small increase, resulting in a much smaller portion compared to the ADC area when the ADC resolution is 8. The CIM (Computing-in-Memory) array area, which is for the crossbar, remains exactly constant since it is not impacted by ADC circuits. As a result, the interconnect circuits (IC) area, although initially smaller than the CIM array, becomes larger than it when the ADC resolution is greater than 7.

#### Iv-B2 ResNet

The total ADC area and chip area exhibit a similar trend as that of AlexNet, but with some notable

Fig. 4: **Binary ResNet inference accuracy vs. first layer input precision — Each line corresponds to a different ADC resolution.**

Fig. 5: **Chip area breakdown for Binary AlexNet with different ADC resolutions — Chip means the total chip area. CIM, IC, ADC, Acc means CIM array area, interconnect circuits area, ADC area, and accumulation circuits area on chip respectively. Other means other peripheries (e.g. decoders, mux, switch matrix, buffers, pooling, and activation units) on the chip. Note that the figure is in log scale.**

\begin{table}
\begin{tabular}{c|c c c c c c} \hline \hline
**Input** & \multicolumn{6}{c}{**ADC Resolution**} \\
**Precision** & 3 & 4 & 5 & 6 & 7 & 8 \\ \hline
1 & 0.24 & 1.89 & 3.34 & 5.76 & 5.14 & 5.40 \\
2 & 0.75 & 22.65 & 20.64 & 32.20 & 40.67 & 40.85 \\
3 & 0.91 & 32.98 & 26.39 & 42.91 & 54.05 & 54.15 \\
4 & 0.84 & 34.82 & 27.04 & 45.49 & 55.42 & 55.88 \\
5 & 0.85 & 35.22 & 27.56 & 46.19 & 55.52 & 56.22 \\
6 & 1.04 & 35.41 & 27.78 & 46.58 & 55.44 & 56.18 \\
7 & 0.85 & 36.07 & 27.47 & 46.49 & 55.58 & 56.04 \\
8 & 0.95 & 35.92 & 27.72 & 46.37 & 55.68 & 55.93 \\ \hline \hline \end{tabular}
\end{table} TABLE III: **Inference accuracy for Binary DenseNet with different input precisions and ADC resolutions — The inference accuracy on GPU with float32 first-layer activation is 56.15%.**
differences. The total CIM array area remains constant, which is roughly half the size of AlexNet's CIM array (2.34 \(mm^{2}\) vs. 4.32 \(mm^{2}\)), despite ResNet having more layers. This is due to AlexNet having three large dense layers (9216 * 4096, 4096 * 4096, 4096 * 1000) at the end, which requires more parameters to be stored in the CIM array. In contrast, ResNet only has one smaller dense layer (512 * 1000) at the end, resulting in fewer parameters to be stored in the CIM array. Consequently, the total IC area of ResNet is larger than the CIM array area from the start when ADC resolution is 3.

#### Iv-B3 DenseNet

Despite having a more intricate and deeper network architecture than ResNet, DenseMet exhibits a larger CIM array area of 4.54 \(mm^{2}\), which is almost twice the size of ResNet's 2.34 \(mm^{2}\) and slightly larger than AlexNet's 4.32 \(mm^{2}\). The total accumulation circuits and other periphery areas scale similar to the other models, yet the total IC area remains only marginally larger than that of ResNet across all ADC resolution levels (2.66 \(mm^{2}\) vs. 2.52 \(mm^{2}\) for ADC resolution = 3 and 8.66 \(mm^{2}\) vs. 8.20 \(mm^{2}\) for ADC resolution = 8). Consequently, the IC area is initially smaller than the CIM array area and surpasses it only when the ADC resolution exceeds 6.

#### Iv-B4 Put It All Together

Table IV lists the precise values of chip area as well as its components as a function of ADC resolution for AlexNet, ResNet, and Densenet. Due to three large dense layers at the end of AlexNet, the chip area of AlexNet, although still smaller than DenseNet, is larger than ResNet, despite the fact that AlexNet is much shallower than the other two. The CIM array area shows a similar pattern, as ResNet is the smallest, and DenseNet is the largest.

Total ADC area is the main factor of increase in chip area for all three models when ADC resolution grows. Similar for all three models, the ADC area grows exponentially (69x) with ADC resolution, and its portion on the chip grows from 13% to 89%. Total IC area, on the other hand, although increasing with ADC resolution, has a very similar value among the three models, which means the area of IC is almost independent of the model structure.

### _Latency_

#### Iv-C1 AlexNet

The per-image inference latency and their components as functions of ADC resolution for Binary AlexNet are shown in Figure 6. The main contributor to the total inference latency is the buffer read latency, which increases from 8.05 \(ms\) to 12.1 \(ms\), resulting in a similar increasing trend in the overall latency.

Although the buffer read latency is larger than 8 ms even at the beginning, the second major component, IC read latency,

\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**BNN** & **ADC** & **Chip area** & **Total CIM array** & **Total IC area** & **Total ADC area** & **Total accumulation** & **Other peripheries** \\
**model** & **resolution** & (\(mm^{2}\)) & (\(mm^{2}\)) & (\(mm^{2}\)) & (\(mm^{2}\)) & **circuits** (\(mm^{2}\)) & (\(mm^{2}\)) \\ \hline AlexNet & 3 & 55.97 & 4.32 & 2.54 & 7.40 & 1.05 & 40.65 \\  & 4 & 64.92 & 4.32 & 2.74 & 15.87 & 1.21 & 40.78 \\  & 5 & 82.62 & 4.32 & 3.10 & 32.79 & 1.37 & 41.04 \\  & 6 & 130.04 & 4.32 & 3.90 & 78.56 & 1.53 & 41.72 \\  & 7 & 236.97 & 4.32 & 5.29 & 182.39 & 1.69 & 43.27 \\  & 8 & 573.53 & 4.32 & 8.26 & 510.94 & 1.85 & 48.15 \\ \hline ResNet & 3 & 32.73 & 2.34 & 2.52 & 4.02 & 0.57 & 23.28 \\  & 4 & 37.67 & 2.34 & 2.72 & 8.60 & 0.65 & 23.35 \\  & 5 & 47.43 & 2.34 & 3.08 & 17.78 & 0.74 & 23.49 \\  & 6 & 73.51 & 2.34 & 3.87 & 42.60 & 0.83 & 23.86 \\  & 7 & 132.11 & 2.34 & 5.25 & 98.90 & 0.91 & 24.70 \\  & 8 & 315.95 & 2.34 & 8.20 & 277.06 & 1.00 & 27.35 \\ \hline DenseNet & 3 & 62.73 & 4.54 & 2.66 & 7.77 & 1.08 & 46.68 \\  & 4 & 72.13 & 4.54 & 2.87 & 16.65 & 1.25 & 46.82 \\  & 5 & 90.70 & 4.54 & 3.25 & 34.41 & 1.41 & 47.09 \\  & 6 & 140.45 & 4.54 & 4.09 & 82.43 & 1.58 & 47.81 \\  & 7 & 252.65 & 4.54 & 5.55 & 191.38 & 1.75 & 49.43 \\  & 8 & 605.80 & 4.54 & 8.66 & 536.12 & 1.91 & 54.56 \\ \hline \hline \end{tabular}
\end{table} TABLE IV: **Chip Area breakdown for different BNN models with different ADC resolutions — Accumulation circuits include adders, shift/Adds on the subarray level, and includes accumulation units on the PE/Tile/Global level. Other peripheries include decoders, mux, switch matrix, buffers, pooling, activation units, etc.**