# HASS: Hardware-Aware Sparsity Search for Dataflow DNN Accelerator

Zhewen Yu1, Sudarshan Sreeram1, Krish Agrawal1, Junyi Wu1, Alexander
Montgomerie-Corcoran1,
Cheng Zhang1, Jianyi Cheng2, Christos-Savvas Bouganis1, Yiren Zhao1  1Imperial
College London, UK, 2University of Cambridge, UK
{zhewen.yu18, sudarshan.sreeram19, krish.agrawal20, junyi.wu21,
alexander.montgomerie-corcoran15,
cheng.zhang122, christos-savvas.bouganis<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Deep Neural Networks (DNNs) excel in learning hierarchical representations
from raw data, such as images, audio, and text. To compute these DNN models
with high performance and energy efficiency, these models are usually deployed
onto customized hardware accelerators. Among various accelerator designs,
dataflow architecture has shown promising performance due to its layer-
pipelined structure and its scalability in data parallelism.

Exploiting weights and activations sparsity can further enhance memory storage
and computation efficiency. However, existing approaches focus on exploiting
sparsity in non-dataflow accelerators, which cannot be applied onto dataflow
accelerators because of the large hardware design space introduced. As such,
this could miss opportunities to find an optimal combination of sparsity
features and hardware designs.

In this paper, we propose a novel approach to exploit unstructured weights and
activations sparsity for dataflow accelerators, using software and hardware
co-optimization. We propose a Hardware-Aware Sparsity Search (HASS) to
systematically determine an efficient sparsity solution for dataflow
accelerators. Over a set of models, we achieve an efficiency improvement
ranging from 1.3$\times$ to 4.2$\times$ compared to existing sparse designs,
which are either non-dataflow or non-hardware-aware. Particularly, the
throughput of MobileNetV3 can be optimized to 4895 images per second. HASS is
open-source: https://github.com/Yu-Zhewen/HASS

##  I Introduction

Deep Neural Networks (DNNs) models are designed to extract relevant features
from raw data, such as images, audio, and text. To improve the performance and
energy efficiency when computing these models, the computation is often mapped
onto hardware accelerators. Among various accelerator architectures, dataflow
accelerators have shown significant performance benefits because of their
deeply pipelined computation between layers [1] and scalable data parallelism
across devices [2].

In hardware accelerator designs, sparsity has been a popular topic, such that
unnecessary computation with zeros can be avoided for better efficiency. There
are two types of sparsity:

  * •

Weight Sparsity: focusing on the zeros in the weights of a model, whose
positions are often available at compile time so that they can be optimized
statically.

  * •

Activation Sparsity: focusing on the zeros inside the intermediate activation
data between layers. The positions of these zeros are only known at run-time,
as they depend on the network input.

With the appropriate hardware support, increased sparsity can lead to fewer
computations. To maximize the sparsity, pruning is a process that simplifies a
model by setting certain weights and activations to zeros, at minimal accuracy
loss based on a set of criteria [3].

$1$$2$$3$$\cdot 10^{-9}$$71$$71.5$$72$HASSPASS [4]HPIPE [5][6]Operations
Density (images/cycle/DSP)Top1 Accuracy (%) Figure 1: HASS explores the
optimal trade-off between classification accuracy and operation density. We
compare HASS with other existing sparse MobileNetV2 implementations.  TABLE I:
Our work is the first attempt to exploring hardware-aware unstructured pruning
for dataflow accelerators. Our approach systematically exploits layer-wise
sparsity in both weights and activations coupled with hardware resource-
constrained analysis.

Approaches | [7] | [8] | [9] | [5] | [6] | [10] | [11] | [4] | Ours
---|---|---|---|---|---|---|---|---|---
Publication Year | 2018 | 2019 | 2020 | 2020 | 2022 | 2023 | 2023 | 2023 | 2024
Dataflow architecture | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓
Weight sparsity | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓
Activation sparsity | ✗ | ✗ | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓
Hardware-aware | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ | ✓

(a) Traditional pruning flow for dataflow accelerator design focuses on
software metrics only, where a pruned model is treated ‘read-only’ in the
hardware end.

(b) Our flow integrates hardware metrics into the pruning search process. We
will explain the details of each step in the highlighted sections.

Figure 2: An overview of our approach. When designing a dataflow accelerator,
traditional pruning steps are separate from the hardware synthesis steps,
which restricts software and hardware co-optimization. Our flow explores both
software and hardware optimization concurrently, opening up opportunities to
find the optimal design.

Traditional pruning approaches only consider software metrics, such as the
overall sparsity in the network, without directly considering the actual
impact on hardware performance. Recently, there has been interest in
developing hardware-aware, co-design pruning approaches for accelerator
designs. These approaches consider hardware performance, such as throughput
and energy efficiency, into the pruning criteria. These approaches realize co-
optimization of both accuracy and hardware performance.

In non-dataflow hardware accelerators, sparsity is exploited to lift the
performance bottleneck in the off-chip memory bandwidth [12]. To efficiently
represent sparse data, encoding techniques are often used to reduce both the
memory footprints and the required off-chip memory bandwidth[13]. Pruning can
be further tailored to optimize individual sparse computation for efficient
data access and processing [14].

However, these approaches cannot be applied to the design of dataflow
accelerators for two reasons. First, the performance bottleneck for dataflow
accelerators is often the computation resources rather than the off-chip
memory bandwidth, because most weights and activations reside on-chip. More
importantly, in a sparse dataflow accelerator, the overall pipeline
performance does not scale proportionally with the total non-zero operations.
For example, increasing the sparsity of a non-slowest layer will not change
the pipeline performance at all, unless it allows for the reallocation of
resources to alleviate the pipeline bottleneck [4].

As such, exploiting the sparsity in a dataflow architecture poses unique
challenges in hardware designs, including scheduling and resource allocation
within a given hardware resource budget. Existing approaches on sparse
dataflow architecture separate the pruning steps and hardware optimization
steps, as shown in Fig. 2(a). In this paper, we propose a systematic approach
to explore unstructured pruning and hardware optimization in a co-design form,
as shown in Fig. 2(b). To the best of our knowledge, our approach is the first
attempt at hardware-aware pruning for dataflow accelerators, as illustrated in
Table II. Our main contributions are as follows.

  * •

The first dataflow accelerator design that exploits both weight sparsity and
activation sparsity coupled with layer-pipelined execution.

  * •

A hardware-aware, unstructured model pruning algorithm that considers both
software pruning metrics (accuracy, sparsity ratio) and hardware performance
(throughput, resource estimations) for systematic optimization.

  * •

Over a set of DNN models, we show that our approach leads to an improved
efficiency ranging from 1.3$\times$ to 4.2$\times$ compared to existing sparse
designs.

##  II Related Work

In the existing studies, the sparse DNN-FPGA accelerators are often
implemented as a single sparse-sparse matrix multiplication engine, shared by
DNN layers in a time-multiplexed manner. As such, the main challenge is
irregular memory access patterns within and between layers. As such, sparse
data are often encoded to save memory space and simplify the scheduling of
non-zero values to processing units [13]. Lu et al. [8] explored the weight
sparsity only. As the positions of zeros are known at compile-time, a look-up
table is built to match the indices of sparse weights and dense activations.
To exploit the dynamic, data-dependent activation sparsity, Zhu et al. [9]
used the activation value to control the clock gating of processing units,
which leads to energy saving but the achieved throughput remains the same as
the dense computation. As irregular sparsity pattern leads to inefficient
hardware, Kong et al. [10] proposed the latency-aware pruning that jointly
optimizes the accuracy and latency. Similarly, Qu et al. [11] modelled energy
consumption and processing cycles, and considered them together in the loss
function, so that the accelerator architecture can be considered.

Instead of scheduling layers in a time-multiplexed manner, HPIPE [5] and PASS
[4] are two works that look in a different direction, building the dataflow
architecture that has multiple pipelined sparse matrix multiplication engines.
To solve the resource allocation and throughput balancing between multiple
sparse engines, they estimated the distribution of sparse data in PyTorch and
used that estimation to guide the process of design space exploration.
However, HPIPE only exploits the weight sparsity, while PASS only exploits
activation sparsity, and neither of them has considered the hardware-aware co-
design in their approaches.

##  III Unstructured Pruning Algorithm

Figure 3: Architecture of the sparse dataflow accelerator. The computation of
layers is pipelined where inside each layer, there are multiple Sparse Vector
Dot Product Engines (SPE) operating in parallel. Inside the SPE, clipping
modules zero out any weight or activation that falls below a configurable
threshold. Zero-filtering then detects these zeros, while the remaining non-
zeros are dispatched to the MACs (implemented with DSPs) via an arbiter. We
also use a dedicated counter to track the number of skipped zeros and manage
accumulation result emissions.

In this paper, we choose to apply one-shot pruning without fine-tuning, where
a significant portion of the network is pruned in a single step, to reduce the
deployment effort. This is in contrast to iterative pruning approaches, where
pruning is performed gradually over multiple iterations, accompanied by fine-
tuning stages [15].

In terms of the pruning criterion, we use the magnitude-based evaluation (L1
norm), where weights and activations having smaller magnitudes below the
threshold will be forced to zeros. The pruning threshold can either be uniform
or unique across layers. Adopting a uniform threshold is straightforward to
explore the trade-off between accuracy and sparsity. However, setting unique
thresholds instead better preserve the network accuracy, as many researchers
have demonstrated the diversity of statistics per layer [16, 14].

Therefore, for each individual layer in the network, our pruning algorithm
requires the identification of unique thresholds $\tau_{w}$ and $\tau_{a}$ for
weight and activation pruning respectively, and the introduced sparsity is
denoted as $S_{w}$ and $S_{a}$, with a range between 0 and 1, indicating the
fraction of zeros in the data. As introduced before, once the pruning
threshold is given, the weight sparsity $S_{w}$ is a fixed value since the
positions of zeros are already available at compile time. However, the
activation sparsity $S_{a}$ depends on the network’s input, making it dynamic
at run-time.

##  IV Sparse Dataflow Architecture

Traditional sparse architectures use two approaches to address the challenges
in memory storage and computational efficiency. For efficient memory storage,
they statically encode weights, trading off between efficient memory storage
and runtime overhead caused by decoding [13]. For efficient computation, they
skip zeros during computation, but leaving the corresponding hardware operator
idle [9]. To fully utilize the available computation resources, we statically
analyze the run-time sparsity and pre-fetch data in a buffer to keep the
hardware operators busy at each cycle. This leads to better performance than
the zero-skipping approach.

Fig. 3 illustrates our target sparse dataflow accelerator architecture. On the
left, the model is presented in a dataflow graph, where each node represents a
dataflow component in hardware and each edge represents their data interface
between every two nodes. The blue nodes are often resource intensive, such as
DSP blocks on an FPGA, and can be significantly optimized by sparse
computation. In each blue node, data parallelism is applied to improve
performance. For example, the middle of the figure represents a hardware
convolutional layer. The hardware optimization steps consider two metrics: 1)
spatial data parallelism by duplicating hardware processing elements, shown as
multiple Sparse Vector Dot Product Engine (SPE) in the figure, and 2) time-
wise folding by iterating using the same SPE and accumulating the result,
shown as ACC and “+” in the figure. Detailed exploration on these metrics are
explained in Section V-A.

The hardware implementation of sparse computation is in SPE, shown in the
right side of Fig. 3. The engine takes a set of inputs from the preceding
layer and weights from the on-chip memory, which are passed through the clip
modules which would zero out any value below the configurable threshold.
Afterwards, zero values are directly forwarded to the counter on the left to
keep track of the iteration count, while non-zero pairs are buffered in a
round-robin arbiter for computation. The arbiter dispatches multiple pairs to
the available Multiply–Accumulate (MAC) units concurrently such that they are
busy in each clock cycle. Once the counter reaches full, the output data will
be released.

Note that the accumulation of the vector product can take place both inside
each SPE and also between multiple SPEs. This strategy can constrain the fan-
in and fan-out of the arbiter, to reduce the area overhead and the degradation
of clock frequency.

The number of the MACs in each SPE, denoted as $N$, is statically determined
by estimating the overall sparsity of the input activations and weights on a
calibration dataset. Consider the scenario where a SPE takes $M$ input weight
pairs. In traditional hardware architecture for dense computation, the
computation would take $M/N$ cycles to complete. Each MAC unit accumulates for
$M/N$ times before emitting partial sums to the $N$-input adder tree for the
final output calculation. In sparse hardware architecture, let $\overline{S}$
be the average sparsity of the input activation and weight pair, where the
computations can be skipped if any s any of them is equal to zero. In this
case, the arbiter will schedule the computation to complete with $t$ cycles,
where:

$t(\overline{S})=\lceil\frac{(1-\overline{S})\times M}{N}\rceil$ (1)

This is also known as initial interval of the SPE. Considering both weight and
activation sparsity, our focus is on the probability of either weight or
activation becoming zero. In our implementation, $N$ and $M$ are customized
for each layer according to its $\overline{S}$.

The proposed sparse computation engine can be aggregated to enhance
performance, both within a layer (intra-layer) and between layers (inter-
layer). Within each convolutional layer, concurrent vector dot products can
occur, allowing parallel computation across the input-channel ($I$) and
output-filter ($O$) dimensions. We designate the levels of this parallelism as
$i\in[1,I]$ and $o\in[1,O]$, respectively. In the case of unstructured
pruning, the sparsity pattern is not uniform within a convolutional layer.
Consequently, the processing rates of $i\times o$ SPEs are dynamic and may be
imbalanced at run-time, potentially causing pipeline stalls. To mitigate this,
we employ the following strategies:

  * •

Balancing Strategy: During compile-time, we estimate the weight and activation
sparsity in each input-channel and output-filter. We then utilize simulated
annealing to solve an allocation problem. This assigns the computation of $I$
input-channels and $O$ output-filters to $i\times o$ sparse computation
engines, minimizing the difference in their processing rates.

  * •

Buffering Strategy: Buffering is employed to absorb the instantaneous variance
of dynamic processing rates. The selection of buffer size involves a trade-off
between resource usage and throughput, and we determine the buffer size
following a heuristic approach similar to [4] which is based on the
observation of moving window statistics.

In the sparse dataflow architecture, computation is pipelined on a layer-by-
layer basis using FIFOs and handshake signals. In this architecture,
activation data is not encoded despite its sparsity. This is because most
intermediate data remains on-chip, and frequent encoding and decoding between
layers would incur significant computational costs.

##  V Hardware-aware Workflow

###  V-A Accelerator Design Space Exploration

This section focuses on the Design Space Exploration (DSE) problem of the
sparse DNN-FPGA accelerator, and it can be formalized using the following
terms:

  * •

$L:\\{l_{0},l_{1},l_{2}...\\}$ denotes the layers in a network;

  * •

$S\subseteq[0,1)$ denotes the sparsity search space;

  * •

$D:\\{d_{0},d_{1},d_{2}...\\}$ denotes all possible hardware design points of
a layer;

  * •

$g\subseteq L\times D\times S$ denotes a design point of the network;

Given a budget for the hardware resources $R$, we search for an efficient $g$
to maximize network throughput in a greedy form.

####  V-A1 Performance modeling

Let $C_{l}$ denote the number of operations (including the zeros) in layer
$l$. The throughput of the same layer is then:

$\theta(l,d,\overline{S})=\frac{i\times o\times M}{C_{l}\times
t(\overline{S})}$ (2)

Since $t$ is dynamic, depending on the average sparsity $\overline{S}$, so as
the throughput of the layer. The sparse dataflow architecture is layer-wise
pipelined, therefore, the network throughput $\theta$ is restricted by the
slowest layer:

$\forall g\ldotp\theta\leq\min_{l\in L}\theta(l,d,\overline{S})$ (3)

The aim of the DSE process is to determine an efficient $g$ with a large
$\theta$.

$1$$2$$3$$4$$5$$6$$7$$8$$9$$10$$11$$12$$13$$14$$15$$16$$0.4$$0.6$$0.8$Sparsity

$1$$2$$3$$4$$5$$6$$7$$8$$9$$10$$11$$12$$13$$14$$15$$16$$2$$4$$6$MAC per SPE

$1$$2$$3$$4$$5$$6$$7$$8$$9$$10$$11$$12$$13$$14$$15$$16$$100$$200$LayerSPE

Figure 4: DSE results of a specific sparse ResNet-18 workload with 16 $3\times
3$ convolutional layers. The allocation of MAC per SPE mainly depends on the
per-layer sparsity statistic. A higher sparsity leads to a smaller MAC per
SPE. However, the increase of the layer index leads to an increasing number of
convolutional filters, so as the number parallel SPEs to match the rate
between layers.

.

####  V-A2 Resource-constrained rate balancing

Apart from the slowest layer, the remaining layers in the pipeline may
underperform, meaning that their actual throughput is significantly lower than
the maximum achievable throughput using the allocated resources. These
hardware resources could be unused at run-time, leading to inefficiency. To
address this, we can reduce the parallelism of these layers without affecting
overall throughput to enable more parallelism at the performance bottleneck.
This is also known as rate balancing.

Let $\theta_{r}(l,d,\overline{S})$ represent the actual throughput of a layer
instance $(l,d,\overline{S})$. After rate balancing, each layer in the
balanced design $g^{\prime}$ is configured with a parallelism level close to
its actual throughput.

$\displaystyle\theta_{l}^{\prime}=\min\\{\theta(l,d^{\prime},\overline{S})|\theta(l,d^{\prime},\overline{S})\geq\theta_{r}(l,d,\overline{S})\\}$
(4) $\displaystyle\forall l\in L\wedge(l,d,\overline{S})\in
g\Rightarrow\theta(l,d^{\prime},\overline{S})=\theta_{l}^{\prime}\wedge(l,d^{\prime},\overline{S})\in
g^{\prime}$ (5)

Constraint (5) ensures that all layers in the design are computing efficiently
in a pipeline, optimizing the overall resource utilization.

####  V-A3 Resource-constrained incrementing

The DSE starts with the resource-minimal design, where the computation within
each layer is fully sequential. At each iteration, the DSE increases the
parallelism of the slowest layer by a small step followed by the rate
balancing process described in step 2). This incremental process repeats until
the resource budget $R$ satisfies. The resource utilization of each sparse
computation engine is modeled on the basis of the regression model.

####  V-A4 Partitioning and reconfiguration

In practice, the restriction caused by finite hardware resources could fail to
map an entire DNN to a single dataflow accelerator device. In this work, we
fold the dataflow pipeline at the block level and iteratively compute them on
the available hardware resources using full reconfiguration on an FPGA device.
This allows changing hardware architecture at run-time at the price of
additional reconfiguration time. To reduce such overhead caused by the
reconfiguration time, the data is processed in a large batch size [1]. The
decisions of where to split the partition and the number of partitions are
given by a simulated annealing solver that trade-off the reconfiguration time
and data parallelism gained.

###  V-B Multi-objective Search

$20$$40$$60$$80$$100$$0.8$$0.9$Number of Iteration Stepsimages/cycle/DSP
($10^{-9}$)Software metrics-onlyWith hardware metrics Figure 5: Comparison
between software metrics-only sparsity search and the proposed hardware-aware
sparsity search for ResNet-18. We use 96 iteration steps for both approaches.

Let’s denote the per-layer weight and activation pruning threshold as
$\tau_{w}$ and $\tau_{a}$. To identify the optimal values of them, we
construct the following optimization problem for the network $L$:

  * •

$f_{acc}$: network accuracy, measured on validation data;

  * •

$f_{spa}$: average sparsity of the network, including both weights and
activations;

  * •

$f_{thr}$: network throughput for a given pruned network searched by DSE;

  * •

$f_{dsp}$: resource utilization of the accelerator, represented by the number
of DSPs used, which is usually the bottleneck in dataflow accelerators.

The objective for the hardware-aware pruning is to maximize accuracy,
sparsity, and throughput while minimizing resource utilization:

$\max_{\\{\tau_{w},\tau_{a}\\},l\in
L}f_{acc}(L)+\lambda_{1}f_{spa}(L)+\lambda_{2}f_{thr}(L)-\lambda_{3}f_{dsp}(L)$
(6)

$\lambda_{1}$, $\lambda_{2}$ and $\lambda_{3}$ are hyperparameters that
normalize the values of these metrics, determined by heuristics. An efficient
search algorithm for multi-objective search is required, and we use Tree-
structured Parzen Estimator (TPE) [17], a Bayesian optimization algorithm that
uses a tree structure to model the probability density of the objective
function and iteratively constructs this tree based on observed evaluations of
the objective function.

The proposed hardware-aware pruning approach could achieve better computation
efficiency (throughput per area) than the same pruning approach with software
metrics only (i.e. accuracy and sparsity). Fig. 5 shows a comparison of these
two approaches for ResNet-18. The proposed hardware-aware search shown as the
green curve is guided by the objective function in Equation 6. The software
metrics-only search shown as the blue curve only uses accuracy and sparsity as
the objective function. Both approaches ran 96 iteration steps, taking about 3
hours to complete. At the initial stage, the change in green curve in
computation efficiency is slow because the objective function is complex and
contains more hardware metrics. Then it reaches a better computation
efficiency because the hardware metrics guided the search towards efficient
hardware implementation. This is more helpful if more iteration steps are
performed for large networks.

ResNet-18ResNet-50MobileNetV2MobileNetV3SMobileNetV3L$0.1$$0.2$$0.5$$1$$2$Y0.0e0]ptY1.0e0]ptY2.0e0]ptY3.0e0]ptY4.0e0]ptY0.0e0]pt3.8$\times$Y1.0e0]pt5.3$\times$Y2.0e0]pt2.7$\times$Y3.0e0]pt2.4$\times$Y4.0e0]pt1.5$\times$images/cycle/DSP
($10^{-9}$)Dense DataflowProposed Sparse Dataflow Figure 6: By exploiting
sparsity in both weights and activations, our approach achieves significant
speedup compared to the dense implementation.

##  VI Evaluation

TABLE II: A comparison with state-of-the-art sparse DNN-FPGA accelerators.

Models | ResNet-18 | ResNet-50 | MobileNetV2 | MobileNetV3S | MobileNetV3L
---|---|---|---|---|---
Works | [4] | Ours | [6] | [4] | Ours | [6] | [5] | [4] | Ours | Dense | Ours | Dense | Ours
Accuracy | 69.75 | 69.59 | N/A | 76.13 | 75.58 | 70.80 | 71.90 | 71.79 | 71.45 | 67.42 | 67.28 | 74.04 | 73.76
Platform | U250 | U250 | 7V690T | U250 | U250 | 7V690T | Stratix10 | U250 | U250 | U250 | U250 | U250 | U250
Freq (MHz) | 250 | 250 | 150 | 250 | 250 | 150 | 390 | 250 | 250 | 250 | 250 | 250 | 250
DSPs | 10974 | 12234 | 2160 | 11952 | 7434 | 2160 | 5928 | 3596 | 5261 | 4282 | 1796 | 6577 | 4324
kLUTs | 1659 | 1679 | 308 | 1721 | 1724 | 308 | 523 | 1552 | 1720 | 971 | 507 | 1535 | 1728
BRAM18k | 4554 | 4817 | 1883 | 4262 | 4178 | 1883 | 4512 | 1774 | 1902 | 2278 | 1779 | 3706 | 5376
images/s | 1904 | 2819 | 33 | 330 | 776 | 302 | 4539 | 1660 | 4495 | 4890 | 4895 | 1897 | 1898
images/cycle/DSP ($10^{-9}$) | 0.69 | 0.92 | 0.10 | 0.11 | 0.42 | 0.93 | 1.96 | 1.84 | 3.42 | 4.57 | 10.90 | 1.15 | 1.76

In terms of the experiment set-up, the FPGA device we used for the result
measurements is AMD Xilinx Alveo U250, and the version of software employed is
Vitis 2023.1. The clock frequency of our accelerators is 250MHz. We examine
our approach on a set of mainstream DNN benchmarks including ResNet-18,
ResNet-50, MobileNetV2 and MobileNetV3, taken from pyTorch torchvision and
trained on the ImageNet dataset. All the networks quantized to 16-bit fixed
point for both weights and activations. Our pruning algorithm is one-shot and
post-training, without applying any fine-tuning to the networks.

The tool flow is presented in Fig. 2(b). We implemented a fully-automated flow
that translates DNN models to Torch FX graph. In an FX graph, the model is
represented as a dataflow graph similar to the one shown on the left of Fig.
3. Our pruning search engine extracts the layer-wise pruning space using the
TPE algorithm. With the proposed hardware model presented in Section V-A, the
hardware cost is evaluated as well as the accuracy loss. This guides the next
iteration step of the pruning search process. The final pruned model and its
hardware parameters are sent to a hardware synthesis tool for hardware
implementation. Our approach is general and can be extended to support other
hardware synthesis tools. For this work, we used a DNN-FPGA synthesis tool
named fpgaConvNet [1] for prototyping.

We compare our approach with other sparse DNN-FPGA accelerators from related
works. Our evaluation encompasses the trade-off among accuracy, resource,
throughput (images/s) and hardware efficiency (images/cycle/DSP). Fig. 6 shows
the improvements from the dense dataflow architecture to the proposed sparse
dataflow architecture, and the detailed results are shown in Table II. We made
the following observations:

  * •

Compared with related work using sparse computation, the dataflow
architectures, including HPIPE[5], PASS [4] and Ours, have significant
performance improvements over the non-dataflow design [6] because of its
deeply pipelined computation. For example, when targeting ResNet-50 and
MobileNetV2, the advantage in terms of throughput per DSP can be up to 4.2
$\times$ and 3.7$\times$, respectively.

  * •

The advantage of sparse dataflow architecture is not always free. Compared to
the non-dataflow sparse accelerator design [6], the dataflow approach requires
more resources (upto 3$\times$ DSPs and 5$\times$ LUTs). This is because the
hardware dataflow pipeline requires extra logic and buffers in exchange for
pipeline performance and data parallelism scalability.

  * •

For MobileNetV3, we also include the dense results in the table. Compared with
the dense results, our sparse implementations achieve the same throughput with
a reduced number of DSP utilized. The throughput remains similar because the
designs are either LUT or BRAM bounded.

  * •

PASS [4] is the work related most to our solution, and it optimizes hardware
for the activation sparsity. However, they do not explore weight sparsity or
consider to drive the pruning strategy with hardware information. Compared
with them, our approach achieves improved efficiency with 1.3$\times$,
3.8$\times$ and 1.9$\times$ on ResNet-18, ResNet-50 and MobileNetV2.
Meanwhile, our post-training accuracy degradation is less than 0.6 percentage
points.

  * •

The variance in the results depends on the sensitivity of the models to data
sparsity. The accuracy loss may be reduced with the help of fine-tuning, at
the price of training time.

##  VII Conclusion

In this paper, we propose a novel approach to exploit software and hardware
co-optimization for sparsity, targeting the dataflow DNN-FPGA accelerator. We
implement a hardware-aware sparsity search that considers both software
pruning metrics and hardware performance for systematic optimization. We also
exploit both static weight sparsity and dynamic activation sparsity for
dataflow accelerator design. Over a set of DNN models, we achieve 1.3$\times$
to 4.2$\times$ efficiency compared to existing sparse designs. In terms of
future work, we will delve into the integration of a more diverse range of
pruning algorithms to further enhance applicability.

## References

  * [1] S. I. Venieris and C.-S. Bouganis, “fpgaconvnet: Mapping regular and irregular convolutional neural networks on fpgas,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 30, no. 2, 2019.
  * [2] Y. Zhang, N. Zhang, T. Zhao, M. Vilim, M. Shahbaz, and K. Olukotun, “Sara: Scaling a reconfigurable dataflow accelerator,” in _2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA)_ , 2021, pp. 1041–1054.
  * [3] A. K. Jain, H. Omidian, H. Fraisse, M. Benipal, L. Liu, and D. Gaitonde, “A domain-specific architecture for accelerating sparse matrix vector multiplication on fpgas,” in _2020 30th International conference on field-programmable logic and applications (FPL)_. IEEE, 2020, pp. 127–132.
  * [4] A. Montgomerie-Corcoran, Z. Yu, J. Cheng, and C.-S. Bouganis, “Pass: Exploiting post-activation sparsity in streaming architectures for cnn acceleration,” in _2023 33rd International Conference on Field Programmable Logic and Applications (FPL)_. IEEE, 2023.
  * [5] M. Hall and V. Betz, “Hpipe: Heterogeneous layer-pipelined and sparse-aware cnn inference for fpgas,” _arXiv preprint arXiv:2007.10451_ , 2020.
  * [6] Z. Liu, Q. Liu, S. Yan, and R. C. Cheung, “An efficient fpga-based depthwise separable convolutional neural network accelerator with hardware pruning,” _ACM Transactions on Reconfigurable Technology and Systems_ , 2023.
  * [7] L. Lu and Y. Liang, “Spwa: An efficient sparse winograd convolutional neural networks accelerator on fpgas,” in _Proceedings of the 55th Annual Design Automation Conference_ , 2018, pp. 1–6.
  * [8] L. Lu, J. Xie, R. Huang, J. Zhang, W. Lin, and Y. Liang, “An efficient hardware accelerator for sparse convolutional neural networks on fpgas,” in _2019 IEEE 27th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)_. IEEE, 2019, pp. 17–25.
  * [9] C. Zhu, K. Huang, S. Yang, Z. Zhu, H. Zhang, and H. Shen, “An efficient hardware accelerator for structured sparse convolutional neural networks on fpgas,” _IEEE Transactions on Very Large Scale Integration (VLSI) Systems_ , vol. 28, no. 9, pp. 1953–1965, 2020.
  * [10] Z. Kong, P. Dong, X. Ma, X. Meng, W. Niu, M. Sun, X. Shen, G. Yuan, B. Ren, H. Tang _et al._ , “Spvit: Enabling faster vision transformers via latency-aware soft token pruning,” in _European Conference on Computer Vision_. Springer, 2022, pp. 620–640.
  * [11] Y. Qu, Y. Ma, and P. Zhou, “A speed-and energy-driven holistic training framework for sparse cnn accelerators,” in _2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)_. IEEE, 2023, pp. 1–6.
  * [12] A. K. Jain, C. Ravishankar, H. Omidian, S. Kumar, M. Kulkarni, A. Tripathi, and D. Gaitonde, “Modular and lean architecture with elasticity for sparse matrix vector multiplication on fpgas,” in _2023 IEEE 31st Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)_. IEEE, 2023, pp. 133–143.
  * [13] A. Parashar, M. Rhu, A. Mukkara, A. Puglielli, R. Venkatesan, B. Khailany, J. Emer, S. W. Keckler, and W. J. Dally, “Scnn: An accelerator for compressed-sparse convolutional neural networks,” _ACM SIGARCH computer architecture news_ , vol. 45, no. 2, pp. 27–40, 2017.
  * [14] S. Dave, R. Baghdadi, T. Nowatzki, S. Avancha, A. Shrivastava, and B. Li, “Hardware acceleration of sparse and irregular tensor computations of ml models: A survey and insights,” _Proceedings of the IEEE_ , vol. 109, no. 10, pp. 1706–1752, 2021.
  * [15] C. M. J. Tan and M. Motani, “Dropnet: Reducing neural network complexity via iterative pruning,” in _International Conference on Machine Learning_. PMLR, 2020, pp. 9356–9366.
  * [16] R. Banner, Y. Nahshan, and D. Soudry, “Post training 4-bit quantization of convolutional networks for rapid-deployment,” _Advances in Neural Information Processing Systems_ , vol. 32, 2019.
  * [17] J. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl, “Algorithms for hyper-parameter optimization,” _Advances in neural information processing systems_ , vol. 24, 2011.


Riemannian 3-manifolds $(Y_{n},g_{n})$ with absolute sectional curvature
$|K_{Y_{n}}|\leq\pi^{2}$, and points $p_{n}\in M_{n}$ such that for all
$n\in\mathbb{N}$,

$|A_{M_{n}}|(p_{n})\cdot\min\\{\mbox{\rm
Inj}_{Y_{n}}(p_{n}),d_{M_{n}}(p_{n},\partial M_{n}),1/2\\}\geq n.$ (7.4)

Consider the open geodesic disk $D_{n}\subset M_{n}$ of center $p_{n}$ and
radius $d_{M_{n}}(p_{n},\partial M_{n})$. Let $p_{n}^{*}\in D_{n}$ be a
maximum of the continuous function

$f_{n}\colon D_{n}\to\mathbb{R},\quad
f_{n}(x)=|A_{M_{n}}|(x)\cdot\min\\{\mbox{\rm
Inj}_{Y_{n}}(x),d_{D_{n}}(x,\partial D_{n}),1/2\\}.$

After passing to a subsequence, we can assume that one of the following three
cases occurs for all $n\in\mathbb{N}$.

  1. (A)

$\min\\{\mbox{\rm Inj}_{Y_{n}}(p_{n}^{*}),d_{D_{n}}(p_{n}^{*},\partial
D_{n}),1/2\\}=\mbox{\rm Inj}_{Y_{n}}(p_{n}^{*})$.

  2. (B)

$\min\\{\mbox{\rm Inj}_{Y_{n}}(p_{n}^{*}),d_{D_{n}}(p_{n}^{*},\partial
D_{n}),1/2\\}=d_{D_{n}}(p_{n}^{*},\partial D_{n})$.

  3. (C)

$\min\\{\mbox{\rm Inj}_{Y_{n}}(p_{n}^{*}),d_{D_{n}}(p_{n}^{*},\partial
D_{n}),1/2\\}=1/2$.

Suppose that case (C) holds. Since $\mbox{\rm Inj}_{Y_{n}}(p_{n}^{*})\geq
1/2$, then Lemma 2.2 in [25] implies:

The injectivity radius function of $B_{Y_{n}}(p_{n}^{*},1/2)$ restricted to
$B_{Y_{n}}(p_{n}^{*},1/8)$ is at least $1/8$.  (7.5)

Applying Theorem 2.1 in [25] to the choices $M=B_{Y_{n}}(p_{n}^{*},1/2)$,
$\Lambda=\pi^{2}$, $\Omega=B_{Y_{n}}(p_{n}^{*},1/10)$,
$\Omega({\delta})=B_{Y_{n}}(p_{n}^{*},1/8)$, $i=1/8$, we conclude that every
point $x\in B_{Y_{n}}(p_{n}^{*},1/10)$ admits harmonic coordinates centered at
$x$ and defined on the geodesic ball $B_{Y_{n}}(x,{\varepsilon}_{0})$ for some
${\varepsilon}_{0}>0$ independent of $x$ and $n$, and the metric $g_{n}$ is
$C^{1,{\alpha}}$-controlled in the sense of Definition 2.2 in terms of a
constant $Q>1$ which is also independent of $n\in\mathbb{N}$.

Let ${\lambda}_{n}=|A_{M_{n}}|(p_{n}^{*})$, which tends to $\infty$ as
$n\to\infty$ because

$\frac{1}{2}|A_{M_{n}}|(p_{n}^{*})=f_{n}(p_{n}^{*})\geq
f_{n}(p_{n})\stackrel{{\scriptstyle(\ref{eq:thm6.6a})}}{{\geq}}n.$ (7.6)

Define
$B_{n}^{\prime}=\left(B_{Y_{n}}(p_{n}^{*},1/10),{\lambda}_{n}^{2}\,g_{n}\right)$.
The sequence of $3$-manifolds $\\{B_{n}^{\prime}\\}_{n}$ converges
$C^{1,{\alpha}}$ to $\mathbb{R}^{3}$ with its standard metric and the harmonic
coordinates in $B^{\prime}_{n}$ centered at $p_{n}^{*}$ converge as
$n\to\infty$ to the usual harmonic coordinates centered at the origin.

Consider the sequence of immersed, one-sided, stable minimal surfaces

$\Delta_{n}=\left(B_{M_{n}}(p_{n}^{*},1/10),{\lambda}_{n}^{2}\,g_{n}\right)\looparrowright
B^{\prime}_{n}.$

Observe that the intrinsic distances in $\Delta_{n}$ from $p_{n}^{*}$ to the
boundary of $\Delta_{n}$ diverge to $\infty$. We claim that the $\Delta_{n}$
have uniformly bounded second fundamental form: Take $x\in
B_{M_{n}}(p_{n}^{*},1/10)$. Since $x\in D_{n}$ because we are in case (C), and
thus

$|A_{M_{n}}|(x)\cdot\min\\{\mbox{\rm Inj}_{Y_{n}}(x),d_{D_{n}}(x,\partial
D_{n}),1/2\\}=f_{n}(x)\leq f_{n}(p_{n}^{*})=\frac{{\lambda}_{n}}{2},$

or equivalently,

$|A_{\Delta_{n}}|(x)\cdot\min\\{\mbox{\rm Inj}_{Y_{n}}(x),d_{D_{n}}(x,\partial
D_{n}),1/2\\}\leq 1/2.$ (7.7)

Observe that $\mbox{\rm Inj}_{Y_{n}}(x)\geq 1/8$ by (7.5). Also,
$d_{D_{n}}(x,\partial D_{n})\geq 2/5$ because $x\in
B_{M_{n}}(p_{n}^{*},1/10)$, $B_{M_{n}}(p_{n}^{*},1/2)\subset D_{n}$ and by the
triangle inequality. Hence, the minimum in the LHS of (7.7) is at least $1/8$,
from where we deduce that $|A_{\Delta_{n}}|(x)\leq 4$, and our claim is
proved.

Therefore, after passing to a subsequence, the $\Delta_{n}$ converge to a
complete minimal surface $S$ immersed in $\mathbb{R}^{3}$ with bounded second
fundamental form, see the arguments at the beginning of Section 5.4 for
details.

We claim that $S$ is stable. If $S$ is two-sided, this is standard, see e.g.
[25, page 636]. We next give a different argument that is valid regardless of
whether $S$ is one- or two-sided. Stability of $S$ in the one-sided case
amounts to show that

$\int_{\widetilde{S}}|A_{\widetilde{S}}|^{2}\phi^{2}\leq\int_{\widetilde{S}}|\nabla\phi|^{2},$
(7.8)

for every compactly supported smooth function $\phi\in
C_{0}^{\infty}(\widetilde{S})$ defined on the two-sided cover $\widetilde{S}$
of $S$ that is anti-invariant, see Definition 2.1. Given such a function
$\phi$, we can view $\phi$ for $n$ sufficiently large as a compactly supported
smooth function $\phi_{n}$ defined on the two-sided cover
$\widetilde{\Delta}_{n}$ of $\Delta_{n}$ that is anti-invariant, and thus, by
stability of $\Delta_{n}$, we have

$\int_{\widetilde{\Delta}_{n}}(|A_{\widetilde{\Delta}_{n}}|^{2}+\mbox{Ric}_{B^{\prime}_{n}}(N_{n},N_{n}))\phi_{n}^{2}\leq\int_{\widetilde{\Delta}_{n}}|\nabla\phi_{n}|^{2},$
(7.9)

where Ric${}_{B^{\prime}_{n}}$ denotes the Ricci curvature of $B^{\prime}_{n}$
and $N_{n}$ is a unit normal vector to $\widetilde{\Delta}_{n}$ in
$B^{\prime}_{n}$. The $C^{1,{\alpha}}$ convergence of the metrics
${\lambda}_{n}\,g_{n}$ to the flat metric on $\mathbb{R}^{3}$ allows us to
take limits in (7.9) as $n\to\infty$ to obtain (7.8), and thus, $S$ is stable.

The desired contradiction (that proves (7.2) in the case that (C) holds) comes
from the fact that there are no complete stable non-flat minimal surfaces in
$\mathbb{R}^{3}$, see Ros [24, Theorem 8].

Next we will explain how to reduce case (A) to case (C). If case (A) holds, we
have $\mbox{\rm Inj}_{Y_{n}}(p_{n}^{*})\leq 1/2$. Let $\mu_{n}=1/\mbox{\rm
Inj}_{Y_{n}}(p_{n}^{*})$. Define
$Y_{n}^{\prime}=\left(Y_{n},\mu_{n}^{2}\,g_{n}\right)$ and
$M^{\prime}_{n}=(M_{n},\mu_{n}^{2}\,g_{n})$. Note that $\mbox{\rm
Inj}_{Y^{\prime}_{n}}(p_{n}^{*})=1$, the absolute sectional curvature of
$Y^{\prime}_{n}$ is less than or equal to
$\frac{\pi^{2}}{\mu_{n}^{2}}\leq\pi^{2}$, which implies we may use the upper
estimate $K_{0}=\pi^{2}$ (in other words, $(M^{\prime}_{n},Y_{n}^{\prime})$ is
a possible counterexample to (7.2) under the normalization introduced in the
second paragraph of this proof), and so $\frac{\pi}{2\sqrt{K_{0}}}=1/2$.
Observe that $(M^{\prime}_{n},Y^{\prime}_{n})$ lies in case (C) because
$d_{M^{\prime}_{n}}(p_{n}^{*},\partial
M^{\prime}_{n})=\mu_{n}\,d_{M_{n}}(p_{n}^{*},\partial M_{n})\geq 1$, and so,
$\min\\{\mbox{\rm
Inj}_{Y^{\prime}_{n}}(p_{n}^{*}),d_{M^{\prime}_{n}}(p_{n}^{*},\partial
M^{\prime}_{n}),1/2)\\}=1/2$. If we check that

$|A_{M^{\prime}_{n}}|(p_{n}^{*})\cdot\min\\{\mbox{\rm
Inj}_{Y^{\prime}_{n}}(p_{n}^{*}),d_{M^{\prime}_{n}}(p_{n}^{*},\partial
M^{\prime}_{n}),1/2)\\}\to\infty$ (7.10)

then we will find a contradiction as we did in case (C). To see this, observe
that two times the LHS of (7.10) can be written as

$|A_{M^{\prime}_{n}}|(p_{n}^{*})=|A_{M^{\prime}_{n}}|(p_{n}^{*})\cdot\mbox{\rm
Inj}_{Y^{\prime}_{n}}(p_{n}^{*})=|A_{M_{n}}|(p_{n}^{*})\cdot\mbox{\rm
Inj}_{Y_{n}}(p_{n}^{*})=f_{n}(p_{n}^{*})\geq n\to\infty,$

which finishes the proof in case (A) occurs. Similar reasoning reduces case
(B) to case (C), which completes the proof of Theorem 7.1. ∎

######  Remark 7.2 (Lower bound estimates for $C^{\prime}_{s}$ and
$C_{s}^{\prime\prime}$).

We claim that $\pi$, $2\pi$ are lower bounds for $C^{\prime}_{s}$,
$C^{\prime\prime}_{s}$, respectively. To see this, consider the Scherk’s
doubly periodic minimal surface $M({\theta})$ in $\mathbb{R}^{3}$,
${\theta}\in(0,\pi/2]$, and its non-orientable, embedded quotient surface
$\widehat{M}({\theta})$ with total curvature $-2\pi$ in the flat quotient
manifold $Y({\theta})=T_{\theta}^{2}\times\mathbb{R}$ where
$T_{\theta}=\mathbb{R}^{2}/\mbox{Span}\\{w_{1}({\theta}),w_{2}({\theta})\\}$,
where

$w_{1}({\theta})=\frac{\pi}{2}\left(\frac{1}{\cos({\theta}/2)},0,0\right),\quad
w_{2}({\theta})=\frac{\pi}{2}\left(0,\frac{1}{\sin({\theta}/2)},0\right).$

Here, the oriented cover $\widetilde{M}({\theta})$ of $\widehat{M}({\theta})$
is conformally $(\mathbb{C}\cup\\{\infty\\})\setminus\\{e^{\pm
i{\theta}/2}\\}$ with Weierstrass data

$\left(g(z)=z,\quad\omega=\frac{i\,dz}{\Pi(z\pm e^{\pm i{\theta}/2})}\right).$

Straightforward calculations show that at $z=0$ in
$(\mathbb{C}\cup\\{\infty\\})\setminus\\{e^{\pm i\pi/4}\\}$ viewed as a point
of $\widehat{M}({\theta})$, the absolute Gaussian curvature is given by
$|K|(0)=16$ and this point is the unique maximum of $|K|$ on
$\widehat{M}({\theta})$. On the other hand, the injectivity radius of
$Y({\theta})$ (at every point) equals $\frac{\pi}{4\cos({\theta}/2)}$ which
has a maximum value of $\frac{\pi}{2\sqrt{2}}$ at ${\theta}=\pi/2$. Therefore,
for any $\theta\in/0,\pi/2]$ we have

$|A_{\widehat{M}({\theta})}|\cdot\mbox{\rm
Inj}_{Y({\theta})}\leq|A_{\widehat{M}(\pi/2)}|(0)\cdot\mbox{\rm
Inj}_{Y(\pi/2)}=|4\sqrt{2}|\frac{\pi}{2\sqrt{2}}=2\pi.$

Hence the constant $C^{\prime\prime}_{s}$ in the above theorem must be at
least $2\pi$.

The standard fundamental region $Q$ for $\widehat{M}(\pi/4)$ in
$\mathbb{R}^{3}$ is a vertical graph bounded by 4 vertical lines and

$|A_{Q}|(0)\cdot d_{Q}(0,\partial Q)=4\sqrt{2}\frac{\pi}{2\sqrt{2}}=2\pi,$

so the constant $C^{\prime}_{s}$ in (7.1) also must be at least $2\pi$. In
fact, $C^{\prime}_{s}$ can be seen to be strictly greater than $2\pi$ by
consideration of the intersection of $M(\theta)$ with a ball of radius
slightly larger than $\frac{\pi}{2\sqrt{2}}$. Therefore, the constant $C_{s}$
given in given in the above theorem also must be greater than $2\pi$.

Next consider the translational quotient of $H$ of a helicoid in
$\mathbb{R}^{3}$ such that $H$ is an embedded, one-sided, stable minimal
surface in $Y=\mathbb{R}^{3}/(\pi\mathbb{Z})$ with finite total curvature
$-2\pi$. Let $p\in H$ be any point on the axis of $H$. Then,

$|A_{H}|\cdot\mbox{\rm Inj}_{Y}\leq|A_{H}|(p)|\cdot\mbox{\rm
Inj}_{Y}(p)=|\sqrt{2}|\frac{\pi}{2}=\frac{\pi}{\sqrt{2}}.$

The slab-type region $W$ of $H$ bounded by two straight lines inside $H$ at
distance $\pi$ apart is stable, and the function $p\in
W\mapsto|A_{W}|(p)d_{W}(p,\partial W)$ has a maximum value at the mid point of
the segment obtained by intersecting the axis of $H$ with $W$, hence

$|A_{W}|(p)\cdot d_{W}(p,\partial W)\leq|A_{W}|(0)\cdot d_{W}(0,\partial
W)=|\sqrt{2}|\frac{\pi}{2}=\frac{\pi}{\sqrt{2}}.$

The above curvature estimates for $\widehat{M}(\pi/2)$, $Q$, $H$ and $W$ lead
us to ask the following question.

######  Question 7.3.

If $M$ is a complete, one-sided, stable minimal surface in a complete flat
3-manifold $Y$, does the following inequality hold?

$\mbox{For all }p\in M,\quad|A_{M}|(p)|\cdot\mbox{\rm Inj}_{Y}(p)\leq 2\pi.$

More generally, does setting $C_{s}^{\prime\prime}=2\pi$ work in Theorem 7.1?

These questions are also motivated by the result by Ros [24] that the only
complete non-flat stable minimal surface in a quotient of $\mathbb{R}^{3}$ by
a rank one (resp. two) group of translations is a quotient of the Helicoid
(resp. quotients of the Scherk doubly periodic minimal surfaces) with total
curvature $-2\pi$.

##  8 Appendix B: Some results in reference [17] used in this paper

In this section we state, for the readers convenience, some results from [17]
that we frequently apply in the proofs of the present paper.

######  Proposition 8.1 (Intrinsic monotonicity of area formula, Proposition
2.4 in [17]).

Let $\overline{B}_{X}(x_{0},R_{1})$ denote a closed geodesic ball in an
$m$-dimensional manifold $(X,g)$, where $0<R_{1}\leq\mbox{\rm
Inj}_{X}(x_{0})$, and suppose that $K_{\mbox{\rm\footnotesize sec}}\leq a$ on
$B_{X}(x_{0},R_{1})$ for some $a\in\mathbb{R}$. Given $H_{0}\geq 0$, define

$R_{0}(a,H_{0})=\left\\{\begin{array}[]{cl}\frac{1}{\sqrt{a}}\mbox{\rm arc
cot}\left(\frac{H_{0}}{\sqrt{a}}\right)&\mbox{if }a>0,\\\ 1/H_{0}&\mbox{if
}a=0\quad\mbox{(if $H_{0}=0$ we take $R_{0}(0,0)=\infty$)}\\\
\frac{1}{\sqrt{-a}}\mbox{\rm arc
coth}\left(\frac{H_{0}}{\sqrt{-a}}\right),&\mbox{if }a<0\quad\mbox{(if
$\frac{H_{0}}{\sqrt{-a}}\geq 1$ we take
$R_{0}(a,H_{0})=\infty$),}\end{array}\right.$ (8.1)

and let $r_{1}=r_{1}(R_{1},a,H_{0})=\min\\{R_{1},R_{0}(a,H_{0})\\}$.

Suppose $M$ is a complete, immersed, connected $n$-dimensional submanifold of
$X$ and $x_{0}\in M$ is a point such that when $\partial M\neq\varnothing$,
$d_{M}(x_{0},\partial M)\geq R_{1}$ and the length of the mean curvature
vector $\vec{H}$ of $M$ restricted to $\overline{B}_{X}(x_{0},R_{1})$ is
bounded from above by $H_{0}$. Then:

  1. 1.

If $M$ is compact without boundary, then there exists $y\in M$ such that the
extrinsic distance from $x_{0}$ to $y$ is greater than or equal to $r_{1}$.

  2. 2.

The $n$-dimensional volume $A(r)$ of $B_{M}(x_{0},r)$ is a strictly increasing
function of $r\in(0,r_{1}]$.

  3. 3.

For all $r\in(0,r_{1}]$ when $r_{1}\neq\infty$ or otherwise, for all
$r\in(0,\infty)$:

$A(r)\geq\left\\{\begin{array}[]{cc}\omega_{n}\,r^{n}e^{-nH_{0}r}&\mbox{ if
$a\leq 0$},\\\ \omega_{n}\,r^{n}e^{-nr(H_{0}+\frac{1}{2}f_{a}(r_{1})r)}&\mbox{
if $a>0$},\end{array}\right.$ (8.2)

where $\omega_{n}$ is the volume of the unit ball in $\mathbb{R}^{n}$ and
given $a>0$, the function $f_{a}\colon[0,\pi/\sqrt{a})\to\mathbb{R}$ is
defined by $f_{a}(t)=\frac{1}{t^{2}}\left[1-t\sqrt{a}\cot(\sqrt{a}t)\right]$,
$t\in[0,\pi/\sqrt{a})$.

######  Corollary 8.2 (Corollary 2.6 in [17]).

Let $R_{1}>0$, $a\in\mathbb{R}$ and $H_{0}\geq 0$, and suppose that $X$ is a
complete Riemannian $m$-dimensional manifold with injectivity radius at least
$R_{1}>0$ and $K_{sec}\leq a$. If $M\looparrowright X$ is a complete, non-
compact immersed $n$-dimensional submanifold with empty boundary and the mean
curvature vector $\vec{H}$ of $M$ satisfies $|\vec{H}|\leq H_{0}$, then $M$
has infinite volume.

######  Proposition 8.3 (Proposition 2.7 in [17]).

Given $R_{1}>0$, $a\in\mathbb{R}$ and $H_{0}\geq 0$, there exists
$r_{2}=r_{2}(R_{1},a,H_{0})\in(0,r_{1}]$ (here $r_{1}$ is given by Proposition
8.1) such that if $X$ is a complete Riemannian 3-manifold with injectivity
radius at least $R_{1}>0$ and $K_{sec}\leq a$, and if $M\looparrowright X$ is
a complete, connected immersed surface with boundary, whose mean curvature
vector $\vec{H}$ satisfies $|\vec{H}|\leq H_{0}$, then for all $p\in\mbox{\rm
Int}(M)$ we have

$\mathrm{Area}[B_{M}(p,r)]\geq 3r^{2},\quad\mbox{whenever
$0<r\leq\min\\{r_{2},d_{M}(p,\partial M)\\}$.}$ (8.3)

Furthermore, given ${\varepsilon}_{0}>0$ define $\displaystyle
C_{A}=\min\\{{\varepsilon}_{0},\frac{r_{2}^{2}}{{\varepsilon}_{0}}\\}$. If
$p\in M$ satisfies $d_{M}(p,\partial M)\geq{\varepsilon}_{0}$, then

$\mathrm{Area}[B_{M}(p,d_{M}(p,\partial M))]\geq C_{A}\,d_{M}(p,\partial M)$
(8.4)

and

$\mathrm{Area}[B_{M}(p,{\varepsilon}_{0})]\geq C_{A}\,{\varepsilon}_{0},$
(8.5)

We finish this summary of auxiliary results taken from [17] with the following
scale-invariant weak chord-arc type estimates for branched minimal surfaces of
finite index in $\mathbb{R}^{3}$.

######  Proposition 8.4 (Proposition 4.1 in [17]).

Given $I,B\in\mathbb{N}\cup\\{0\\}$, let
$f\colon(\Sigma,p_{0})\looparrowright(\mathbb{R}^{3},\vec{0})$ be a complete,
connected, pointed branched minimal surface with index at most $I$ and total
branching order at most $B$. Given $R>0$, let $\Omega_{R}$ denote the
component of $f^{-1}(\overline{\mathbb{B}}(R))$ that contains $p_{0}$. Then,
the following scale-invariant estimates hold and depend only on $I$, $B$:

  1. 1.

For any $p\in\Omega_{R}$,

$d_{\Omega_{R}}(p,\partial\Omega_{R})<\widehat{L}R,$ (8.6)

where $\widehat{L}=\sqrt{\frac{1}{2}(3I+2B+3)}$.

  2. 2.

If $f$ is injective with image a plane, then the distance between any two
points of $\Omega_{R}$ is less than or equal to $2R$. Otherwise, given points
$p,q$ in $\Omega_{R}$,

$d_{\Omega_{2R}}(p,q)<\widehat{C}R,$ (8.7)

where
$\widehat{C}=\widehat{C}(I,B)=8\widehat{L}^{3}+2\pi\widehat{L}^{2}-20\widehat{L}-\frac{\pi}{2}$.
In particular, $\Omega_{R}\subset B_{\Sigma}(p,\widehat{C}R)$ for every
$p\in\Omega_{R}$.

William H. Meeks, III at<EMAIL_ADDRESS>
Mathematics Department, University of Massachusetts, Amherst, MA 01003 Joaquín
Pérez at<EMAIL_ADDRESS>
Department of Geometry and Topology and Institute of Mathematics (IMAG),
University of Granada, 18071, Granada, Spain

## References

  * [1] N. S. Aiex and H. Hong.  Index estimates for surfaces with constant mean curvature in 3-dimensional manifolds.  Calc. Var. Partial Differential Equations, 60(3), 2019.  MR4176856, Zbl 1460.53052.
  * [2] L. Ambrozio, R. Buzano, A. Carloto, and B. Sharp.  Geometric convergence results for closed minimal surfaces via bubbling analysis.  Calc. Var. Partial Differential Equations, 61(25), 2021.  MR4350199, Zbl 1486.53015, https://doi.org/10.1007/s00526-021-02135-x.
  * [3] T. Bourni, B. Sharp, and G. Tinaglia.  CMC hypersurfaces with bounded Morse index.  J. reine angew. Math., 786(3):175–203, 2022.  MR4434752, Zbl 07538648, DOI 10.1515/crelle-2022-0009.
  * [4] R. Buzano and B. Sharp.  Qualitative and quantitative estimates for minimal hypersurfaces with bounded index and area.  Trans. Amer. Math. Soc., 370(6):4373–4399, 2018.  MR3811532, Zbl 1390.53008.
  * [5] O. Chodosh, D. Ketover, and D. Maximo.  Minimal hypersurfaces with bounded index.  Invent. math., 209(3):617–664, 2017.  MR3681392, Zbl 1378.53072.
  * [6] O. Chodosh and D. Maximo.  On the topology and index of minimal surfaces II.  To appear in J. Differential Geometry, preprint at https://arxiv.org/pdf/1808.06572.pdf.
  * [7] T. H. Colding and W. P. Minicozzi II.  The space of embedded minimal surfaces of fixed genus in a $3$-manifold IV; Locally simply-connected.  Ann. of Math., 160:573–615, 2004.  MR2123933, Zbl 1076.53069.
  * [8] T. H. Colding and W. P. Minicozzi II.  The space of embedded minimal surfaces of fixed genus in a $3$-manifold V; Fixed genus.  Ann. of Math., 181(1):1–153, 2015.  MR3272923, Zbl 06383661.
  * [9] D. Fischer-Colbrie.  On complete minimal surfaces with finite Morse index in $3$-manifolds.  Invent. Math., 82:121–132, 1985.  MR0808112, Zbl 0573.53038.
  * [10] M. Grüter.  Regularity of weak ${H}$-surfaces.  J. Reine Angew. Math., 329:1–15, 1981.  MR0636440, Zbl 0461.53029.
  * [11] E. Hebey and M. Herzlich.  Harmonic coordinates, harmonic radius and convergence of Riemannian manifolds.  Rendiconti di Matematica, Serie VII, 17:569–605, 1997.  MR1620864, Zbl 0912.53011.
  * [12] L. Jorge and W. H. Meeks III.  The topology of complete minimal surfaces of finite total Gaussian curvature.  Topology, 22(2):203–221, 1983.  MR0683761, Zbl 0517.53008.
  * [13] M. Karpukhin.  On the Yang-Yau inequality for the first Laplace eigenvalue.  Geom. Funct. Anal., 29(6):1864–1885, 2019.  MR4034923, Zbl 1429.58040.
  * [14] F. J. López and A. Ros.  Complete minimal surfaces of index one and stable constant mean curvature surfaces.  Comment. Math. Helv., 64:34–53, 1989.  MR0982560, Zbl 0679.53047.
  * [15] D. Maximo.  A note on minimal surfaces with bounded index.  To appear in Comm. Anal. Geom., https://arxiv.org/abs/1812.10728.
  * [16] W. H. Meeks III.  The classification of complete minimal surfaces with total curvature greater than $-8\pi$.  Duke Math. J., 48:523–535, 1981.  MR0630583, Zbl 0472.53010.
  * [17] W. H. Meeks III and J. Pérez.  Geometry of branched minimal surfaces of finite index.  Preprint at https://arxiv.org/abs/2211.03529.pdf.
  * [18] W. H. Meeks III and J. Pérez.  Geometry of CMC surfaces of finite index.  To appear in ANS series. Preprint at https://doi.org/10.48550/arXiv.2212.14428.
  * [19] W. H. Meeks III, J. Pérez, and A. Ros.  The Dynamics Theorem for properly embedded minimal surfaces.  Math. Ann., 365(3–4):1069–1089, 2016.  MR3521082, Zbl 06618524.
  * [20] W. H. Meeks III, J. Pérez, and A. Ros.  Local removable singularity theorems for minimal laminations.  J. Differential Geom., 103(2):319–362, 2016.  MR3504952, Zbl 06603546.
  * [21] W. H. Meeks III and G. Tinaglia.  The geometry of constant mean curvature surfaces in $\mathbb{R}^{3}$.  To appear in JEMS, preprint at http://arxiv.org/pdf/1609.08032v1.pdf.
  * [22] W. H. Meeks III and G. Tinaglia.  Triply periodic constant mean curvature surfaces.  Adv. Math., 335(7):809–837, 2018.  MR2836679, Zbl 1396.53087.
  * [23] M. Micallef and B. White.  The structure of branch points in minimal surfaces and in pseudoholomorphic curves.  Ann. of Math., 141(1):35–85, 1995.  MR1314031, Zbl 0873.53038.
  * [24] A. Ros.  One-sided complete stable minimal surfaces.  J. Differential Geom., 74:69–92, 2006.  MR2260928, Zbl 1110.53009.
  * [25] H. Rosenberg, R. Souam, and E. Toubiana.  General curvature estimates for stable $H$-surfaces in $3$-manifolds and applications.  J. Differential Geom., 84(3):623–648, 2010.  MR2669367, Zbl 1198.53062.
  * [26] A. B. Saturnino.  On the Genus and Area of Constant Mean Curvature Surfaces with Bounded Index.  J. Geom. Anal., 31:11971–211987, 2021.  MR4322559, Zbl 1486.53076.