# MAFIA: Machine Learning Acceleration on FPGAs for IoT Applications

Nikhil P Ghanathe1 Vivek Seshadri2 Rahul Sharma2 Steve Wilton1 Aayan Kumar3

###### Abstract

Recent breakthroughs in ML have produced new classes of models that allow ML
inference to run directly on milliwatt-powered IoT devices. On one hand,
existing ML-to-FPGA compilers are designed for deep neural-networks on large
FPGAs. On the other hand, general-purpose HLS tools fail to exploit properties
specific to ML inference, thereby resulting in suboptimal performance. We
propose MAFIA, a tool to compile ML inference on small form-factor FPGAs for
IoT applications. MAFIA provides native support for linear algebra operations
and can express a variety of ML algorithms, including state-of-the-art models.
We show that MAFIA-generated programs outperform best-performing variant of a
commercial HLS compiler by 2.5$\times$ on average.

##  I Introduction

Traditionally, IoT devices are used for data collection and the data analysis
is performed in the cloud [1, 2, 3]. However, performing ML inference directly
on IoT devices has recently gained attention, offering benefits such as real-
time analysis, increased privacy, and reduced energy consumption. Recent
breakthroughs have produced new classes of ML applications that have compute
and memory requirements low enough to be run directly on milliwatt-scale IoT
devices [4, 5] while still having high classification accuracy. While
microcontrollers are often used to implement these applications [6, 7],
reconfigurable devices like FPGAs may be better suited for such applications.
Several reconfigurable architectures for edge computing have been proposed by
prior works [8, 9, 10]. On one hand, as we can configure an FPGA to directly
run a program, it can deliver superior performance and energy efficiency
compared to general-purpose micro-controllers. On the other hand, unlike ASICs
which do not allow updates to algorithms once deployed and incur a high non-
recurring engineering (NRE) cost, an FPGA can be reprogrammed in the field
using over-the-air updates [11, 12] to accommodate the constant evolution of
ML models. Unfortunately, FPGAs are notoriously hard to program, even for
experts.

To enable practical use of FPGAs in this domain, we need a compiler that can
take high-level specifications of a ML model and directly compile it to an
FPGA program. Unfortunately, existing solutions are insufficient in that they
mainly focus on running DNN models on large FPGAs. However, state-of-the-art
ML models [4, 5] for IoT applications are mostly based on classical ML
algorithms [13], and not DNNs. In fact, existing DNN-to-FPGA compilers cannot
even express the state-of-the-art ML models designed for IoT applications, and
the techniques used by DNN-to-FPGA compilers do not extend to small form-
factor FPGAs because they assume an abundance of FPGA resources that allows
them to preconfigure the FPGA as, for instance, a sea of matrix multipliers.
While general-purpose C-HLS tools can express our target models, they do not
exploit specific properties of ML inference and thereby generate suboptimal
programs. Section II describes the limitations of these two approaches in more
detail.

We propose MAFIA, a compiler for Machine-learning Acceleration on FPGA for IoT
Applications. MAFIA provides native support for linear algebra operations and
can express classical ML inference algorithms [13] like decision trees [14]
and k-nearest neighbors [15]. We currently support a subset of TensorFlow [16]
and the framework of a recent prior work, SeeDot [17] that compiles ML
inference code to IoT devices (Section III-A). We compare MAFIA with four
different tools: A) Bambu HLS, B) Xilinx Vivado HLS, C) Vivado HLS with
automatically generated compiler hints, and D) (C) with additional manual
hints. We evaluate these mechanisms on two state-of-the-art ML models [4, 5],
each on ten different data sets used as benchmark by prior works [5, 4, 17].
MAFIA-generated programs consistently outperform all prior approaches –
2.5$\times$ better on average compared to Vivado HLS with automatically
generated hints.

##  II Limitations of Existing HLS Tools

###  II-A ML-HLS tools target the Neural-network family

Existing ML-HLS tools such as DNN Weaver [18], FP-DNN [19] and DeepBurning
[20] specifically target deep neural network (DNN) workloads. There are
several reasons these tools do not work well for our target models.

Building Blocks: These tools focus on DNN layers (e.g. conv, pooling, FC). As
a result, currently, they cannot express classical algorithms such as decision
tree, k-Nearest Neighbors, and kernels like RBF of SVM. Although, we can
incorporate support for the classical algorithms in these frameworks, the
optimization strategies for DNN’s are at odds with classical algorithms (see
below). MAFIA identifies that matrix operations are the fundamental building
blocks of most classical ML algorithms and expresses the input program as a
matrix data flow graph (DFG).

Parallelism: The logical view of a DNN model is serial. Therefore, ML-HLS
tools only exploit intra-layer parallelism since opportunities for inter-layer
parallelization are often scarce. MAFIA exploits both intra-node and inter-
node parallelism (Section IV-A).

Nature of Computation: Most DNN workloads are computation-bound. Therefore,
the overhead of data shuffling across layers has minimal impact on latency
[21]. However, classical ML algorithms are memory-bound, and data shuffling
can significantly increase latency. MAFIA manages this by introducing
intelligent design constraints (Section IV-A).

Resource-efficiency: Most ML-HLS tools focus primarily on optimizing matrix
multiplication. However, in ML models for IoT applications, the dominant
kernel can be any node in the matrix DFG. For example, in the output code of
MAFIA, a matrix addition node may be more critical for latency than a matrix
multiplication node. MAFIA iteratively optimizes the design based on the
criticality of each operation in the program (Section IV-E). Furthermore,
existing tools usually target high-end FPGAs and assume abundance of
resources. As a result, the optimization techniques employed in these tools
are impractical for resource-scarce devices.

###  II-B Limited Scope for Optimizations of C-based HLS tools

Difficult to extract parallelism: Automatically extracting parallelism from a
sequential C program is difficult. Most safe compiler transformations make
conservative assumptions and yield suboptimal performance. To extract better
performance, some HLS tools allow the programmer to annotate the C program
with compiler hints. However, this requires expert knowledge, is restrictive
in the kind of information that can be provided and is time-consuming.

Difficult to explore solution space: One approach explored by prior work is to
automatically generate loop unrolling hints for the HLS compiler. To determine
the best unrolling factor for each loop, the compiler must estimate how
critical each loop is for the program and the impact of unrolling on resource
consumption. This is hard because 1) finding the critical path in a C program
requires knowledge of target domain 2) HLS compilers have poor accuracy in
predicting (without requiring synthesis and simulation) the resource
consumption and critical path latency (Section VI-B).

##  III Overview of MAFIA

We propose MAFIA, a compiler that generates high-performance Verilog program
from a high-level specification of an ML inference algorithm for a milliwatt-
scale FPGA. The heart of the compiler is an optimizer which minimizes the
latency of the critical path in the DFG by _determining the level of
parallelism with which each node in the program’s DFG should be executed_. We
formulate this optimization problem as an integer programming problem (Section
IV-E).

We augment MAFIA with two components. The first component is a library of
Verilog templates with one template for each type of matrix operation. Each
template is Verilog implementation of the operation, parameterized by the
dimensions of the input matrices and a _parallelism factor_ (PF). Our Matrix
Template Library currently supports various dense and sparse matrix
operations: sparse matrix-vector multiplication (SpMV), matrix
addition/subtraction, dot product, outer product, hadamard product, dense
matrix-vector/vector-matrix/matrix-matrix multiplication, scalar-matrix
multiplication and non-linear activations including exponentiation, relu,
sigmoid and hyperbolic-tan. These operations are sufficient to express a
variety of ML models targeting IoT applications [22], including state-of-the-
art models [4, 5]. MAFIA’s optimizer is agnostic to the exact implementation
of the template. Therefore, we can improve performance further by providing an
improved template for a particular operation or extend support to other matrix
operations by simply adding to the library. The second component is a set of
latency/resource estimation models that can predict the amount of FPGA
resources and execution latency for an operation given its input dimensions
and PF. Since there are finite number of operation types, generating these
templates and their regression models is tractable and a one-time-effort
during the tool development.

###  III-A MAFIA Compiler Flow

Figure 1 shows the compiler’s flow. We provide an overview in this section;
more details are provided in Section IV.

InputProgramDFGGeneratorMatrixDFGPF=1ProfilerDFG with PF=1Latency &
ResourceEstimatesBest-PFEstimatorDFG withBest-PF
EstimatesScheduleGeneratorDFGwithScheduleVerilogGeneratorVerilogCodeLatency/ResourceEstimation
ModelsParameterized MatrixTemplate Library

Figure 1: Overview of MAFIA Flow

Given an input program, the _DFG generator_ extracts the matrix DFG of the
computation along with the matrix dimensions. The _PF-1 Profiler_ profiles the
resource consumption and latency of each node in the DFG when their PF is set
to 1. This information is used by the estimation models to estimate the
resource consumption and latency of each node. The _Best-PF Estimator_ then
uses our optimizer to find the best PF for each node that minimizes the
overall latency of the program. The _Scheduler Generator_ uses the static DFG
to generate a schedule that executes the program in _data flow order_. This
allows MAFIA to execute data-independent nodes in parallel. Finally, the
_Verilog Generator_ uses the identified PFs and the template library to
generate the final Verilog code.

##  IV Detailed Design

In this section, we describe MAFIA in detail. We begin by describing the
_Paramterized Matrix Template Library_ and the _Latency/Resource Estimation
Models_ (yellow blocks in Figure 1). We then explain the flow of the MAFIA
compiler.

###  IV-A Parameterized Matrix Template Library

At the backend of the MAFIA compiler is a library of hand-optimized Verilog
templates, one for each type of matrix operation. Each template consists of
two components: 1) the execution unit and 2) the data interface unit. The
execution unit is the core computation unit (eg. adder, multiply-accumulate)
with a configurable number of processing elements (PE) specified by the PF.
The data interface unit holds logic for supplying data to the execution unit
by reading the output memories of the node’s predecessors and writing the
result generated by the execution unit into the input memories of the node’s
consumers. The structure of the data interface unit depends both on the
execution PF of the node and the PF of the consumer nodes as follows.

Consider two nodes: producer and consumer. In the producer node, each PE
writes its partial output to a buffer. However, if the execution PF of
producer (i.e. number of PEs) does not match that of the consumer, two sets of
buffers with data shuffling logic in between may be needed. For nodes that can
complete their execution in linear time, e.g., matrix addition, this data
shuffling eliminates any performance benefit gained by parallelizing its
execution.

To avoid this problem, we introduce additional constraints on PFs (shown in
Figure 2). We classify nodes as: 1)  _linear-time nodes_ , those that can
complete execution in linear time (or less) in terms of the input size (e.g.,
matrix addition), and 2)  _non-linear-time nodes_ , those that take worse than
linear time to complete execution (e.g., matrix multiplication). We associate
each node with multiple PFs: one for each input, one for its execution unit,
and one for its output. The figure shows a two-input linear-time node and a
two-input non-linear-time node. For a linear-time node, we require the input,
execution, and the output PFs to be the same (thereby avoiding the need for
any data shuffling). For non-linear time nodes, we introduce a logic before
and after the execution to appropriately shuffle the data. Finally, for two
nodes with a producer-consumer relationship, the output PF of the producer
must be equal to the input PF of the consumer. One natural implication of
these constraints is that a sub-graph of connected linear-time nodes will all
have the same PFs. We exploit this observation further to enable pipelined
execution of linear-time nodes (Section IV-G).

###  IV-B Latency/Resource Estimation Models

The key optimization problem in MAFIA is to determine the best PF for each
node in the DFG such that the overall latency is minimized, while fitting
inside the resource budget of the FPGA. This entails exploration of a vast
solution space of possible PF assignments. To reduce exploration time, we
build regression models for each template that can predict the resource
consumption and latency of a node. The overall resource consumption of a
candidate solution is the sum of the resource consumption of all nodes and the
execution latency is the sum of the latency of all the nodes in the _critical_
path (path with maximum latency). For our target models, we find that the
buffering is done mainly using distributed RAM (LUTRAM) because of smaller
matrix dimensions. As a result, there are typically more than enough memory
resources (BRAM, Flip Flops) on our target FPGA boards, so we focus on
predicting only the compute resources (LUT+LUTRAM, DSP).

We build a separate estimation model for each type of matrix operation. Given
the input dimensions of the operation and a target PF, our model estimates the
resource consumption (LUTs and DSPs) and execution latency of the operation.
In general, we expect LUT consumption to increase linearly with PF. The DSP
usage in a PE is known statically for an operation. We expect latency of a
node, in general, to reduce linearly as $1/\textrm{PF}$. However, we find that
for some operations, the parallel execution is followed by some linear
reduction of the partial sums (e.g., DotProduct). Accordingly, we use the
following models for estimating LUT, DSP, and latency consumption of a node.
In the equations below, LUT[1] and Latency[1] are the LUT consumption and
latency of the corresponding operation with the specified input dimensions
when the PF is set to 1.

$\displaystyle\textrm{Latency}[\textrm{PF}]$ $\displaystyle=$
$\displaystyle(\alpha_{L}+\beta_{L}\cdot\textrm{PF}+\gamma_{L}\cdot\frac{1}{\textrm{PF}})\cdot\textrm{Latency}[1]$
$\displaystyle\textrm{LUT}[\textrm{PF}]$ $\displaystyle=$
$\displaystyle(\alpha_{LUT}+\beta_{LUT}.\textrm{PF})\cdot\textrm{LUT}[1]$
$\displaystyle\textrm{DSP}[\textrm{PF}]$ $\displaystyle=$
$\displaystyle\alpha_{DSP}\cdot\textrm{PF}$

These equations require values for $\alpha_{LUT}$, $\beta_{LUT}$,
$\alpha_{DSP}$, $\alpha_{L}$, $\beta_{L}$, and $\gamma_{L}$ for each matrix
type. Of these, $\alpha_{DSP}$ is set by the template developer to the number
of DSPs used by the template’s PE. We use a training algorithm to find the
remaining parameters. To do this, we generate multiple sets of training data
points. Within each set, we fix the input dimensions to an arbitrary value and
vary the PF from 1 to a number beyond which the operation cannot be further
parallelized by the underlying template. We synthesize and simulate Verilog
implementations for each set to obtain the true LUT consumption and execution
latency for each operation. We then identify the best parameters that minimize
the mean squared error on the training data set. This is an one-time-effort
for a family of FPGAs (Artix-7 in our case). These regression models are pre-
trained during the tool development and are already included as a part of the
MAFIA framework.

iPF1iPF2ePFoPFiPF1 = iPF2 = ePF = oPF Linear-TimeNodeiPF1iPF2ePFoPFData
Shuffling Logic(no constraints)Non-Linear-TimeNodeoPF = iPFInter-
nodeConnection

Figure 2: Linear-time and non-linear-time nodes

###  IV-C Data Flow Graph Generator

MAFIA constructs the data flow graph (DFG) of the input program by analyzing
the data dependencies in the program. Each node in the DFG is annotated with
1) type of operation, 2) input dimensions for the operation, and 3) any static
model parameters for the operation. MAFIA currently includes a DFG generator
for SeeDot DSL [17]. We also support a subset of Tensorflow [16] by converting
the Tensorflow program to SeeDot and extracting the DFG.

###  IV-D PF-1 Profiler

The PF-1 profiler determines LUT[1] and Latency[1] described earlier. For each
node in the program’s DFG, it synthesizes the Verilog implementation of the
corresponding template with the node’s operation dimensions to obtain the
value of LUT[1] (once for each node). It then simulates the whole design with
a random input to measure the PF=1 latency (one-time). The PF-1 profiler tags
each node of the DFG with the measured values and passes the DFG to the next
stage.

###  IV-E Best-PF Estimator

The goal of the Best-PF Estimator is to determine the best PF assignment for
each node. The best PF of a node depends on the criticality of the node in the
input program, which is influenced by the node’s data dimensions. For example,
across the 20 datasets we evaluate in Section VI, the PF for the SpMV node
generated by the MAFIA optimizer ranges from 3 to 71. This further
demonstrates the inefficiency of existing DNN accelerators that optimize for a
single type of workload.

Since all the execution PFs are integers, the optimization problem can be
expressed as an integer program. We explore two optimization strategies for
the best-PF estimator.

####  IV-E1 Black-box optimization

In this approach we employ a generic solver to solve the optimization problem.
The critical path of the program can change as PF assignments change.
Therefore, we express this optimization metric as a min-max problem: minimize
the maximum latency across all paths in the program. Our framework adds
constraints for each path in the DFG, stating that the sum of latency of all
the nodes for each path should be less than a target latency. The integer
program is then setup to minimize this target latency. Since solving an
integer program is NP-hard, we relax it to be an optimization problem over
real numbers and round the PFs to the nearest integer.

####  IV-E2 Greedy Optimization

As finding a global optimum can be computationally intensive, we also explore
a greedy strategy. Our greedy algorithm initializes the PFs of all the nodes
to one. It then repeats the following steps while resources are still
available.

  1. 1.

Identify the current critical path of the program.

  2. 2.

Identify the most critical node in that path. The most critical node is the
one which yields the maximum _benefit_ when its PF is increased by one subject
to the constraints established in Section IV-A. Note that when a node’s PF is
increased by one, the optimizer may also have to increase the PF of some
connected nodes.

  3. 3.

If the optimizer cannot increase the PF of any node on the critical path, it
exits immediately. As MAFIA executes the program in data flow order, there is
no benefit to parallelizing a non-critical node even if there are resources
available.

  4. 4.

If not, MAFIA increases the PF of the critical node by one and iterates.

We support two _benefit_ metrics: reduction in latency and reduction in
latency per additional LUT consumed.

###  IV-F Scheduler Generator

MAFIA enables concurrent execution of nodes wherever possible. Each template
is associated with a _start_ and _done_ signal. A node can start execution as
soon as its _start_ signal is asserted. When a node completes execution, it
asserts its _done_ signal. MAFIA generates the controller that encodes this
scheduling logic for the program’s DFG.

###  IV-G Pipelining Linear-Time Nodes

When consecutive operations have the same PF, it is possible to view these two
nodes as a super node and pipeline their execution (rather than waiting for
the first to complete before starting the second). This optimization
eliminates the need for memory buffers between pipelined nodes. To perform
this optimization, MAFIA identifies clusters of linear-time nodes that are
connected to each other and pipelines them. The pipeline begins execution only
when _all_ the nodes supplying input to the pipeline have completed execution.

##  V Evaluation Methodology

###  V-A Hardware Setup and Benchmarks

We target the small form-factor Xilinx Arty-board, which has 20800 logic units
(LUTs), 90 DSP (multiplier) slices and 225 KB of on-chip memory operating at
10MHz. For comparison with microcontrollers, we use the results presented by
SeeDot [17]. SeeDot targets the Arduino Uno Board (8-bit ATmega328P) with 2KB
of SRAM and 32KB of read-only flash operating at 16MHz.

We evaluate different tools using two state-of-the-art machine learning
algorithms: Bonsai [5], a decision-tree based classifier, and ProtoNN [4], a
k-nearest-neighbour based classifier, from Microsoft’s EdgeML library [22].
These algorithms are crafted specifically to run on milliwatt-scale IoT
devices and have state-of-the-art accuracies on various tasks. We use ten
standard ML datasets (both binary and multiclass): cifar [23], character
recognition (cr) [24], usps [25], mnist [26], letter [27] and ward [28]. In
total, we evaluate 20 different DFGs as our benchmarks.

###  V-B Comparison Points

We compare MAFIA with four mechanisms that can currently be used for
generating Verilog programs from high-level specifications for our target
models. We leverage the quantization scheme from a prior work, SeeDot [17].
The fixed-point programs generated from SeeDot are used in our evaluations.
Bambu HLS and Vivado HLS We run Bambu with the BAMBU-PERFORMANCE-MP flag (-O3
optimizations) and Vivado HLS with the default options without any additional
compiler hints (denoted by Vivado No Opt. in Figure 3).

Vivado Auto Opt. In this variant, we compare our work against a recent work,
SeeDot [17]. SeeDot is a language and a compiler to generate efficient
implementation of ML inference algorithms targeting low-end IoT devices. The
FPGA backend of SeeDot 1) uses a hand-optimized Verilog implementation for
Sparse Matrix Vector Multiplication (the most time consuming kernel in the
microcontroller-based implementations of [4, 5]) with a fixed parallelism
factor of 10 and 2) accelerates the rest of the program by automatically
annotating the input C program to Vivado HLS with loop unrolling and
pipelining hints.

Vivado + MAFIA. To understand the best we could do using a commercial HLS
tool, we use the parallelism factors obtained from the MAFIA optimizer to
generate appropriate hints to the Vivado HLS compiler. We build this variant
on top of Vivado Auto Opt. First, we set the parallelism factor for the hand-
optimized SpMV implementation to the PF value of the SpMV node generated by
the MAFIA optimizer. Second, we iteratively incorporate the other PFs from the
MAFIA optimizer by appropriately unrolling the outermost loop in the C-code of
the corresponding operation. We then manually improve the design performance
by further unrolling the loops of the program, until the solution runs out of
resource budget.

MAFIA. In our evaluations, we extract the DFGs from the SeeDot [17] framework
and use them as input for MAFIA. All our main results are presented with the
best performing configuration in which the code is generated by our greedy
optimizer (see Section VI-C for more details) with latency reduction per
additional LUT as the benefit metric.

Vivado No OptygVivado Auto OptygVivado +
MAFIAygMAFIAyg24816326412825651210242048CIFAR-BCR-BMNIST-BUSPS-BWARD-BCR-
MCURET-MLETTER-MMNIST-MUSPS-MCIFAR-BCR-BMNIST-BUSPS-BWARD-BCR-MCURET-MLETTER-
MMNIST-MUSPS-MMeanBonsaiProtoNNPrediction Latency (uS)

Figure 3: Prediction latency using different compilers (lower is better).
Y-axis is in log-scale.

##  VI Results

LUTygDSPygLUTRAMygFFygBRAMyg20%40%60%80%100%VivadoHLSVivadoAuto OptVivado
+MAFIAMAFIAResource Utilization

Figure 4: FPGA resource utilization

We first compare MAFIA-generated programs with other prior approaches on both
prediction latency and resource utilization (Section VI-A). Our results show
that MAFIA generates programs that reduce prediction latency by 2.5$\times$ on
average compared to the best previous approach. Figure 4 illustrates the
efficiency of MAFIA in allocating resources based on criticality of nodes.
This is underlined in our evaluation in Section VI-B, where we observe that
our approach is more accurate than existing tools. Finally, in Section VI-C,
our evaluation of two optimization strategies, greedy and black-box
optimization, reveal that the greedy approach is significantly faster and
generates programs of similar quality (if not better) compared to the black-
box approach.

###  VI-A Comparison of Different Mechanisms

Our results show that all FPGA tools perform comfortably better than programs
running on microcontrollers. Vivado No Opt. performs 14$\times$ better than
microcontroller implementations. However, as expected MAFIA outperforms both
Bambu HLS and Vivado No Opt significantly. Therefore, in the rest of the
paper, we present results on the following mechanisms: 1) Vivado Auto Opt, 2)
Vivado + MAFIA, and 3) MAFIA.

Figure 3 plots the prediction latency of these mechanisms for all twenty
benchmarks. The y-axis is in log-scale. Table I lists the number of features
for each dataset and their baseline (microcontroller) latencies. Figure 4
shows the average utilization of LUT, DSP, LUT RAM, Flip Flops, and Block
RAMs. Since we find memory resources not to be a bottleneck, we focus our
attention on LUT and DSP.

####  VI-A1 Vivado Auto Opt.

Vivado Auto Opt improves prediction latency by 7$\times$ on average and
quadruples the utilization of compute resources compared to Vivado No Opt. by
using a hand-optimized implementation for SpMV and adding automatic loop
unrolling hints. However, this prior approach is still lacking in two aspects.
First, it uses a fixed parallelism factor of 10 for SpMV on all data sets.
But, we observe that the criticality of this operation varies significantly
with different data sets. As detailed in Section IV-E, the PF for SpMV
generated by the MAFIA optimizer ranges from 3 to 71. Second, the mechanism
used by prior work to estimate resource utilization had high error rate and
results in subpar loop unrolling hints.

####  VI-A2 Vivado + MAFIA.

Vivado + MAFIA involves two steps. First, we incorporate the PFs generated by
MAFIA’s optimizer in the code generated by Vivado Auto Opt by appropriately
setting the PF of the SpMV node and the loop unrolling factors for the other
nodes. Although, this improved the performance by 27% on average, we observed
that this approach still resulted in suboptimal performance. This is because,
_Best-PF Estimator_ of MAFIA assumes that the program will be executed in data
flow order. However, Vivado HLS does not execute independent nodes in
parallel. Therefore, even parallelizing the non-critical nodes in the DFG can
enable better performance. We manually unrolled loops for non-critical nodes
as well till we hit the resource budget. With this improvement, Vivado + MAFIA
outperforms Vivado Auto Opt by up to 3.5$\times$ on average across all
datasets.

DATASET |  Num Features | Bonsai | ProtoNN
---|---|---|---
Baseline Latency(us) |  Baseline Latency(us)
cifar-b | 400 | 6121 | 14112
cr-b | 400 | 6263 | 28446
mnist-b | 784 | 11568 | 15983
usps-b | 256 | 4099 | 9206
ward-b | 1000 | 14733 | 23241
cr-m | 400 | 29030 | 34667
curet-m | 610 | 39731 | 37769
letter-m | 16 | 11161 | 35377
mnist-m | 784 | 16026 | 18491
usps-m | 256 | 9140 | 14017

TABLE I: Baseline (microcontroller) latency and number of features for the
evaluated datasets

####  VI-A3 MAFIA.

Our analysis of the performance and resource consumption of the different
mechanisms show that MAFIA improves upon the state-of-the-art on two fronts.
First, MAFIA is able to better estimate required parallelism factors for each
node. Second, MAFIA executes the program in data flow order, rather than
sequentially. MAFIA outperforms Vivado Auto Opt by 4.2$\times$ and even Vivado
+ MAFIA (with manual hints) by 2.5$\times$ despite consuming only half the LUT
resources compared to Vivado + MAFIA. This highlights the fact that the
performance improvement in C-based HLS tools is only due to intra-node
parallelism. With static knowledge of the program’s data flow graph, MAFIA is
able to schedule operations in data flow order, and thereby execute
independent nodes in parallel. Unfortunately, there are no simple hints that
can be provided to Vivado HLS to express this opportunity. Also, the hand-
optimized implementation of each matrix operation template allows MAFIA to
more efficiently perform the underlying arithmetic operations. Therefore,
MAFIA exploits both inter-node and intra-node parallelism and can offer higher
benefits on more resource constrained FPGAs.

###  VI-B MAFIA Resource/Latency Estimation Models

The accuracy of MAFIA’s estimation models ensures that the optimizer does not
overestimate resources and yields suboptimal results by setting lower PFs. The
final error of MAFIA estimation models is 36% for LUT, 17% for DSPs, and 99%
for overall latency. The high error in latency estimate is in part due to the
fact that our estimation algorithm does not capture the pipelining
optimization (Section IV-G). Pipelining results in significant reduction in
latency while consuming similar amount of compute resources. However, the
latency model correctly captures the relative latencies of all nodes, which is
sufficient to guide the MAFIA-optimizer. In any case, to put these numbers in
context, the estimation error of the Vivado HLS compiler for programs
generated by Vivado + MAFIA is 73% for LUT, 673% for DSP, and fails to provide
latency estimates.

###  VI-C Greedy vs Black-box optimization

So far, we have presented results using our greedy optimization strategy. As
described in Section IV-E1, we also explore the use of a generic solver. We
observe that the prediction latency of programs generated by the greedy
approach are on average 10% _lower_ than those obtained from the black-box
approach (for Bonsai across all data sets). The greedy optimizer is also
_significantly_ faster than the black-box approach (22$\times$ on average).
The better performance of the greedy approach can be attributed to our
rounding mechanism in the black-box optimization. To ensure that we fit within
the resource budget of the FPGA, we round down all the PF numbers obtained
from the black-box solver. Optimal rounding is itself an NP-hard problem.

Power Consumption: The average power consumption across all implementations
(20 datasets) is 76.15 mW.

##  VII Related Work

Development of small, accurate ML models for IoT devices is a budding research
area [5, 4, 29, 30] with several novel and interesting applications including
face-detection, assistance for visual impaired, remote farming and radar
classification. We believe MAFIA can allow these applications to be run on
similar form-factor FPGA instead of microcontrollers, thereby providing
significant boost in the performance of the underlying models. Many _ML-HLS
tools_ [31, 32, 33, 19, 18, 34, 35, 36] exist to compile ML inference to FPGAs
from popular frameworks like Tensorflow and Pytorch. Unfortunately, all of
them target DNN workloads and focus their attention on high-end FPGAs [37, 38,
39, 40]. Other works on automatically extracting parallelism in general-
purpose HLS tools include [41, 42, 43]. The closest work to MAFIA is SeeDot
[17], a framework that compiles efficient code for microcontrollers and low-
end FPGAs from a high-level specification.

##  VIII Conclusion

ML inference models targeting milli-watt powered IoT devices depart from the
DNN architecture and use sophisticated primitives with fewer parameters [4,
5]. However, most current HLS for ML research focuses on neural networks and
target high-end FPGAs, resulting in tools that lack the requisite
expressiveness. Thus, developers in the IoT space have to rely on general
purpose HLS tools that have sufficient expressiveness but suffer from poor
resource utilization. MAFIA achieves both high expressiveness and efficiency;
it is a fully automatic toolchain that compiles device-agnostic ML algorithms
to efficient Verilog code. We have evaluated MAFIA on state-of-the-art ML
algorithms in the IoT space and it outperforms prior work by $2.5\times$ on
average.

##  IX Acknowledgements

This work was funded by NSERC through the COHESA strategic network.

## References

  * [1] Zhanlin Ji et al.  “A cloud-based car parking middleware for IoT-based smart cities: Design and implementation”  In _Sensors_ 14.12 Multidisciplinary Digital Publishing Institute, 2014, pp. 22372–22393
  * [2] Jayavardhana Gubbi, Rajkumar Buyya, Slaven Marusic and Marimuthu Palaniswami  “Internet of Things (IoT): A vision, architectural elements, and future directions”  In _Future generation computer systems_ 29.7 Elsevier, 2013, pp. 1645–1660
  * [3] Moeen Hassanalieragh et al.  “Health monitoring and management using Internet-of-Things (IoT) sensing with cloud-based processing: Opportunities and challenges”  In _2015 IEEE international conference on services computing (SCC)_ , 2015, pp. 285–292  IEEE
  * [4] Chirag Gupta et al.  “ProtoNN: Compressed and accurate kNN for resource-scarce devices”  In _International Conference on Machine Learning_ , 2017, pp. 1331–1340
  * [5] Ashish Kumar, Saurabh Goyal and Manik Varma  “Resource-efficient Machine Learning in 2 KB RAM for the Internet of Things”  In _International Conference on Machine Learning_ , 2017, pp. 1935–1944
  * [6] Shishir G. Patil et al.  “GesturePod: Enabling On-Device Gesture-Based Interaction for White Cane Users”  In _Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology_ , UIST ’19  New Orleans, LA, USA: Association for Computing Machinery, 2019, pp. 403–415  DOI: 10.1145/3332165.3347881
  * [7] Deepak Vasisht et al.  “FarmBeats: An IoT Platform for Data-Driven Agriculture”  In _Networked Systems Design and Implementation (NSDI)_ USENIX, 2017  URL: https://www.microsoft.com/en-us/research/publication/farmbeats-iot-platform-data-driven-agriculture/
  * [8] G. Korol et al.  “MCEA: A Resource-Aware Multicore CGRA Architecture for the Edge”  In _2020 30th International Conference on Field-Programmable Logic and Applications (FPL)_ , 2020, pp. 33–39  DOI: 10.1109/FPL50879.2020.00017
  * [9] C. Tan et al.  “LOCUS: Low-power customizable many-core architecture for wearables”  In _2016 International Conference on Compliers, Architectures, and Sythesis of Embedded Systems (CASES)_ , 2016, pp. 1–10  DOI: 10.1145/2968455.2968506
  * [10] C. Tan, M. Karunaratne, T. Mitra and L. Peh  “Stitch: Fusible Heterogeneous Accelerators Enmeshed with Many-Core Architecture for Wearables”  In _2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA)_ , 2018, pp. 575–587  DOI: 10.1109/ISCA.2018.00054
  * [11] T. Gomes et al.  “Towards an FPGA-based network layer filter for the Internet of Things edge devices”  In _2016 IEEE 21st International Conference on Emerging Technologies and Factory Automation (ETFA)_ , 2016, pp. 1–4  DOI: 10.1109/ETFA.2016.7733684
  * [12] Jinhwan Jung, Jihoon Ryoo, Yung Yi and Song Min Kim  “Gateway over the Air: Towards Pervasive Internet Connectivity for Commodity IoT”  In _Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services_ , MobiSys ’20  Toronto, Ontario, Canada: Association for Computing Machinery, 2020, pp. 54–66  DOI: 10.1145/3386901.3388949
  * [13] Tom M. Mitchell  “Machine learning, International Edition”, McGraw-Hill Series in Computer Science  McGraw-Hill, 1997  URL: https://www.worldcat.org/oclc/61321007
  * [14] J.. Quinlan  “Induction of Decision Trees”  In _Mach. Learn._ 1.1 USA: Kluwer Academic Publishers, 1986, pp. 81–106  DOI: 10.1023/A:1022643204877
  * [15] T. Cover and P. Hart  “Nearest neighbor pattern classification”  In _IEEE Transactions on Information Theory_ 13.1, 1967, pp. 21–27
  * [16] Martı́n Abadi et al.  “TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems”  In _CoRR_ abs/1603.04467, 2016  URL: http://arxiv.org/abs/1603.04467
  * [17] Sridhar Gopinath, Nikhil Ghanathe, Vivek Seshadri and Rahul Sharma  “Compiling KB-sized Machine Learning Models to Tiny IoT Devices”  In _Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation_ , PLDI 2019, 2019, pp. 79–95
  * [18] Hardik Sharma et al.  “From High-level Deep Neural Models to FPGAs”  In _The 49th Annual IEEE/ACM International Symposium on Microarchitecture_ , MICRO-49, 2016, pp. 17:1–17:12
  * [19] Y. Guan et al.  “FP-DNN: An Automated Framework for Mapping Deep Neural Networks onto FPGAs with RTL-HLS Hybrid Templates”  In _2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)_ , 2017, pp. 152–159  DOI: 10.1109/FCCM.2017.25
  * [20] Y. Wang et al.  “DeepBurning: Automatic generation of FPGA-based learning accelerators for Neural Network family”  In _53nd ACM/EDAC/IEEE Design Automation Conference_ , 2016, pp. 1–6  DOI: 10.1145/2897937.2898002
  * [21] N.. Jha and S. Mittal  “Modeling Data Reuse in Deep Neural Networks by Taking Data-Types into Cognizance”  In _IEEE Transactions on Computers_ , 2020, pp. 1–1  DOI: 10.1109/TC.2020.3015531
  * [22] Dennis, Don Kurian and Gopinath, Sridhar and Gupta, Chirag and Kumar, Ashish and Kusupati, Aditya and Patil, Shishir G and Simhadri, Harsha Vardhan  “EdgeML: Machine Learning for resource-constrained edge devices”  URL: https://github.com/Microsoft/EdgeML
  * [23] Alex Krizhevsky  “Learning multiple layers of features from tiny images”, 2009
  * [24] Teófilo Emı́dio Campos, Bodla Rakesh Babu and Manik Varma  “Character Recognition in Natural Images”  In _VISAPP 2009 - Proceedings of the Fourth International Conference on Computer Vision Theory and Applications, Lisboa, Portugal, 2009, Vol. 2_ , 2009, pp. 273–280
  * [25] Jonathan J. Hull  “A database for handwritten text recognition research”  In _IEEE Transactions on pattern analysis and machine intelligence_ 16.5 IEEE, 1994, pp. 550–554
  * [26] Yann LeCun, Léon Bottou, Yoshua Bengio and Patrick Haffner  “Gradient-based learning applied to document recognition”  In _Proceedings of the IEEE_ 86.11 IEEE, 1998, pp. 2278–2324
  * [27] Chih-Wei Hsu and Chih-Jen Lin  “A comparison of methods for multiclass support vector machines”  In _IEEE transactions on Neural Networks_ 13.2 IEEE, 2002, pp. 415–425
  * [28] Jingjing Yang et al.  “Group-sensitive multiple kernel learning for object categorization”  In _Computer Vision, 2009 IEEE 12th International Conference on_ , 2009, pp. 436–443  IEEE
  * [29] Don Dennis, Chirag Pabbaraju, Harsha Vardhan Simhadri and Prateek Jain  “Multiple Instance Learning for Efficient Sequential Data Classification on Resource-constrained Devices” slides/DennisPSJ18.pdf  In _Proceedings of the Thirty-first Annual Conference on Neural Information Processing Systems (NeurIPS)_ , 2018, pp. 10976–10987  URL: all_papers/DennisPSJ18.pdf
  * [30] Aditya Kusupati et al.  “FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network” slides/fastgrnn.pdf  In _Proceedings of the Thirty-first Annual Conference on Neural Information Processing Systems (NeurIPS)_ , 2018, pp. 9031–9042  URL: all_papers/KusupatiSBKJV18.pdf
  * [31] D. Holanda Noronha, K. Gibson, B. Salehpour and S… Wilton  “LeFlow: Automatic Compilation of TensorFlow Machine Learning Applications to FPGAs”  In _2018 International Conference on Field-Programmable Technology (FPT)_ , 2018, pp. 393–396
  * [32] S. Mouselinos et al.  “TF2FPGA: A Framework for Projecting and Accelerating Tensorflow CNNs on FPGA Platforms”  In _2019 8th International Conference on Modern Circuits and Systems Technologies (MOCAST)_ , 2019, pp. 1–4
  * [33] J. Duarte et al.  “Fast inference of deep neural networks in FPGAs for particle physics”  In _Journal of Instrumentation_ 13.7, 2018, pp. P07027  DOI: 10.1088/1748-0221/13/07/P07027
  * [34] X. Zhang et al.  “DNNBuilder: an Automated Tool for Building High-Performance DNN Hardware Accelerators for FPGAs”  In _2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD)_ , 2018, pp. 1–8
  * [35] J. Fowers and et. al.  “A Configurable Cloud-Scale DNN Processor for Real-Time AI”  In _2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA)_ , 2018, pp. 1–14  DOI: 10.1109/ISCA.2018.00012
  * [36] Shehzeen Hussain et al.  “FastWave: Accelerating Autoregressive Convolutional Neural Networks on FPGA”  In _IEEE/ACM International Conference on Computer-Aided Design (ICCAD)_ IEEE, 2019  DOI: 10.1109/iccad45719.2019.8942122
  * [37] X. Lian et al.  “High-Performance FPGA-Based CNN Accelerator With Block-Floating-Point Arithmetic”  In _IEEE Transactions on Very Large Scale Integration (VLSI) Systems_ 27.8, 2019, pp. 1874–1885
  * [38] Y. Sun, B. Liu and X. Xu  “An OpenCL-Based Hybrid CNN-RNN Inference Accelerator On FPGA”  In _2019 International Conference on Field-Programmable Technology (ICFPT)_ , 2019, pp. 283–286
  * [39] Martin Hardieck, Martin Kumm, Konrad Möller and Peter Zipf  “Reconfigurable Convolutional Kernels for Neural Networks on FPGAs”  In _Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays_ , FPGA ’19  Seaside, CA, USA: Association for Computing Machinery, 2019, pp. 43–52  DOI: 10.1145/3289602.3293905
  * [40] Duncan J.M Moss et al.  “A Customizable Matrix Multiplication Framework for the Intel HARPv2 Xeon+FPGA Platform: A Deep Learning Case Study”  In _Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays_ , FPGA ’18, 2018, pp. 107–116
  * [41] Wei Zuo et al.  “Improving High Level Synthesis Optimization Opportunity through Polyhedral Transformations”  In _Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays_ , FPGA ’13, 2013, pp. 9–18
  * [42] G. Natale et al.  “A polyhedral model-based framework for dataflow implementation on FPGA devices of Iterative Stencil Loops”  In _2016 IEEE/ACM International Conference on Computer-Aided Design (ICCAD)_ , 2016, pp. 1–8
  * [43] J. Liu, J. Wickerson, S. Bayliss and G.. Constantinides  “Polyhedral-Based Dynamic Loop Pipelining for High-Level Synthesis”  In _IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems_ 37.9, 2018, pp. 1802–1815


# Iron Oxide Nanoparticles as a Contrast Agent for Synchrotron Imaging of
Sperm

Mette Bjerg Lindhøj  Department of Computer Science, University of Copenhagen,
Copenhagen, Denmark.  Danish Research Centre for Magnetic Resonance, Center
for Functional and Diagnostic Imaging and Research, Copenhagen University
Hospital Hvidovre, Hvidovre, Denmark<EMAIL_ADDRESS>Susan Rudd Cooper
Department of Chemistry and Nano-Science Center, University of Copenhagen,
Copenhagen, Denmark.  Andy S. Anker  Department of Chemistry and Nano-Science
Center, University of Copenhagen, Copenhagen, Denmark.  Anne Bonnin  Paul
Scherrer Institut, Villigen PSI, Switzerland.  Mie Kristensen  Department of
Pharmacy, University of Copenhagen, Copenhagen, Denmark.  Klaus Qvortrup
Department of Biomedical Science/CFIM, University of Copenhagen, Copenhagen,
Denmark.  Kristian Almstrup  Department of Growth and Reproduction, Copenhagen
University Hospital - Rigshospitalet, Copenhagen, Denmark.  Kirsten M. Ø.
Jensen  Department of Chemistry and Nano-Science Center, University of
Copenhagen, Copenhagen, Denmark.  Tim B. Dyrby  Danish Research Centre for
Magnetic Resonance, Center for Functional and Diagnostic Imaging and Research,
Copenhagen University Hospital Hvidovre, Hvidovre, Denmark.  Department of
Applied Mathematics and Computer Science Technical University of Denmark,
Kongens Lyngby Denmark.  Jon Sporring  Department of Computer Science,
University of Copenhagen, Copenhagen, Denmark.

###### Abstract

Fast phase-contrast imaging offered by modern synchrotron facilities opens the
possibility of imaging dynamic processes of biological material such as cells.
Cells are mainly composed of carbon and hydrogen, which have low X-ray
attenuation, making cell studies with X-ray tomography challenging. At
specific low energies, cells provide contrast, but cryo-conditions are
required to protect the sample from radiation damage. Thus, non-toxic
labelling methods are needed to prepare living cells for X-ray tomography at
higher energies. We propose using iron oxide nanoparticles due to their proven
compatibility in other biomedical applications. We show how to synthesize and
attach iron oxide nanoparticles and demonstrate that cell-penetrating peptides
facilitate iron oxide nanoparticle uptake into sperm cells. We show results
from the TOMCAT Nanoscope (Swiss Light Source), showing that iron oxide
nanoparticles allow the heads and midpiece of fixed sperm samples to be
reconstructed from X-ray projections taken at $10\text{\,}\mathrm{keV}$.

##  1 Introduction

Synchrotron X-ray tomography (SXRT) is a vital tool in scientific research of
biological samples offering insight into diverse domains such as axon and
muscle fibre morphology [1, 2] and the biomechanics of insects [3, 4, 5].
Combining the spatial resolution to view axons with the temporal resolution
needed to see the dynamics of a blowfly offers excellent possibilities for
studying the biomechanics of single cells.

Achieving dynamic tomography of single cells is dependent on two essential
factors: High spatio-temporal resolution and high sample contrast at a low
absorbed dose.

The recent development of brilliant synchrotron light sources and fast
detector systems have made it possible to capture much faster motion and a
wider range of sizes than ever seen before [6]. Within the last decade, the
effective pixel size has been pushed to the micro-and nanometer scale with
sub-second exposure times [6, 3, 8, 9, 10, 11, 12, 13, 14, 7].

Mammalian sperm are very suitable candidates for probing the limits of SXRT
for dynamic single-cell imaging as these are primary mammalian cells readily
available and among the most dynamic cells known. Furthermore, changes in
sperm motility caused by environmental chemicals acting on CatSper, the
primary calcium channel of sperm, has been proposed to be a potential
mechanism involved in recent decreasing fertility trends [15, 16]. Dynamic
single-cell studies of sperm is highly relevant for understanding parameters
important for fertility.

The image intensity value in an X-ray projection comes mainly from a reduction
of the intensity of the X-ray beam as it passes through the material, known as
attenuation. The attenuation is caused by absorption or scattering of the beam
and is affected by factors such as the atomic properties of the material and
the beam energy. At most energies, cells and water-like substances, such as
cell media needed for cells to survive, have very similar attenuation
properties. This makes SXRT of living cells challenging as they provide almost
no X-ray contrast against their surroundings. The water window is a unique
spectral region where the attenuation difference between carbon and oxygen is
especially large [17], allowing cells to be clearly distinguished from a water
background [18]. However, the increased absorption also increases the
radiation dose deposited in the cells. Therefore, vitrification of the sample
is required to protect the sample from radiation damage during X-ray
tomography in the water window, making the study of living cell dynamics
impossible.

Different materials can be added to cells to achieve sufficient contrast in
X-ray imaging at higher energies where the absorbed dose is less. One such
method is the addition of electron-dense osmium tetroxide to myelin for
enhanced contrast of axons [1]. While osmium also provides good contrast in
sperm, it is highly toxic, making it unfit for living cells. Other known
contrast agents for X-ray imaging are iodine or gadolinium-based drugs, used
for contrast enhancement during, e.g., angiography [19, 20] in medical CT
scans. Gold nanoparticles can also provide contrast enhancement during X-ray
imaging [21, 22, 23]. Unfortunately, gold nanoparticles have adverse effects
on the motility of sperm, leaving approximately one-quarter of the cells
immotile after treatment [24].

Iron oxide nanoparticles are widely used in biomedicine due to their high
biocompatibility and magnetic properties. Furthermore, they enter sperm
through endocytosis without affecting their motility or ability to fertilize
an egg [25]. Iron oxide attenuates the beam more than cells, making it a
promising candidate for enhanced contrast in SXRT of biological samples such
as sperm.

When using nanoparticles for contrast enhancement during SXRT, the number of
particles taken up by the cell, and their ability to distribute throughout the
cell, are essential to provide sufficient contrast. To aid the uptake of the
nanoparticles, cell-penetrating peptides (CPP), such as Penetratin, can be
used [26]. CPPs comprise a family of short peptides (up to approximately 30
amino acids), which inherently permeates cell membranes. Several CPPs,
including Penetratin, are classified as cationic/amphipathic [27]. They
internalize into cells via direct membrane translocation or endocytosis and
have demonstrated the ability to facilitate uptake of co-incubated cargo such
as nanoparticles into various cell types [28]. Penetratin has earlier
demonstrated efficient internalization into sperm without compromising their
cell viability or motility [29].

_We performed experiments to investigate whether iron oxide nanoparticles
could be used as a non-toxic alternative to existing staining methods for
achieving the necessary spatial resolution to image sperm using SXRT. To
develop the staining method, we synthesized iron oxide nanoparticles covered
in polyvinyl alcohol and incubated them with boar sperm, using Penetratin to
facilitate the uptake. We used transmission electron microscopy to verify the
attachment of the nanoparticles to the sperm and obtained SXRT volumes of
fixed sperm stained with iron oxide nanoparticles at the TOMCAT beamline (
Swiss Light Source (SLS)) to assess the success of the staining._

##  2 Results

###  2.1 The synthesized nanoparticles consisted of a magnetite core coated in
PVA

Figure 1: PXRD analysis. The graph shows a comparison of PXRD data of the
solid nanoparticles before being thoroughly dried (dark blue), a PVA solution
(medium blue), water (light blue), and a calculated scattering pattern of a
published magnetite structure (green) [30]. The data are collected by
measuring the scattering intensity at different scattering angles
($\mathrm{2\theta}$). The peaks from the scattering pattern of our synthesized
nanoparticles (dark blue) coincide with the peaks from the PVA and iron oxide
scattering patterns (medium blue and green) and are not present in the water
background, indicating that the particles are iron oxide nanoparticles covered
in PVA.

To make the non-toxic iron oxide nanoparticle contrast agent for boar sperm,
we synthesized iron oxide nanoparticles coated in polyvinyl alcohol (PVA) by
modifying a previously reported co-precipitation method [31], see Materials
and Methods. Then, to verify the method’s success, we used Powder X-ray
diffraction (PXRD) to estimate the size of the particles and verify their
crystal structure. Powder X-ray Diffraction (PXRD) is used to identify and
characterize the structure of crystalline materials. The PXRD peaks contain
information on the atomic arrangement in materials.

Figure 1 shows the X-ray diffraction pattern of the nanoparticles compared to
the calculated X-ray scattering patterns of magnetite [30]. The X-ray
scattering pattern from the nanoparticle suspension shows Bragg reflections
(peaks) at values of 2$\theta$ equal to $12.7^{\circ}$, $15.3^{\circ}$,
$18.8^{\circ}$, $20.0^{\circ}$ and $21.8^{\circ}$, which are also in the
calculated magnetite pattern. In addition, in the X-ray scattering pattern
from the nanoparticle suspension, we see large, broad peaks centred at
$2\theta$ equal to $7.0^{\circ}$, $8.0^{\circ}$ and a peak at $2\theta$ equal
to $14.5^{\circ}$, which are also present in the X-ray scattering pattern from
the PVA solution, but not in the X-ray scattering pattern from the water
background. This suggests that PVA is in the particle suspension, and we
expect it to cover the surface of the nanoparticles. Using the Scherrer
equation, equation (1) in Materials and Methods, we estimated the crystallite
size of the particles to be approximately $6\text{\,}\mathrm{nm}$ in diameter.
Conclusively, PXRD analysis confirms that iron oxide nanoparticles with a size
of approx. 6 nm have been synthesized, and that they are most likely covered
by PVA. Combined with the small size of the particles, they seem to be a
promising candidate for an X-ray contrast agent.

###  2.2 Uptake of nanoparticles by sperm

Figure 2: TEM images of cells stained with nanoparticles. The figure
illustrates how we believe the incubation worked. By incubating the sperm with
iron oxide nanoparticles and Penetratin, the iron oxide nanoparticles
internalized into the cell membrane (blue line). On the left is an overview of
the sample, showing that the sperm are relatively close together, which is a
possible cause for our inability to segment the tails in the synchrotron
volumes. In the centre circle, the particle placement is shown. The particles
were found both in the heads and tails as labelled in the image. The placement
was determined by the texture of the image. On the right, we show a close up
of the particles, where we see that several particles group together, forming
lumps.

To promote uptake of the iron oxide nanoparticles, we incubated a density
gradient purified sperm sample with a suspension of the synthesized
nanoparticles. The nanoparticles were incubated with the purified sperm using
Penetratin to aid the uptake of the particles; see Materials and Methods.
Then, the sperm samples were fixed and imaged with transmission electron
microscopy (TEM) to verify that the particles had entered the sperm. The TEM
images verified that the electron-dense magnetite nanoparticles had entered
the cells (Figure 2). We determined the location of the particles to be inside
the heads, midpieces and tails of the cells by investigating the texture of
the cells in the EM-images. Traces of particles outside the cells were also
observed.

###  2.3 Heads and midpieces can be segmented from the synchrotron data

(A) Typical duroc sperm

(B) Synchrotron reconstructions and segmentations

Figure 3: Synchrotron results. A: An illustration of a Duroc sperm with
relevant mean measures. B: A tabular illustration showing the differences
between unstained sperm (left column) and sperm stained with iron oxide
nanoparticles (right column). In the unstained sample (left column) we were
not able to find any shapes reminiscent of sperm, only blobs we believe to be
dense DNA inside the sperm heads. When the sperm were stained (right column)
with our iron oxide nanoparticles the heads and midpieces could be segmented,
which was verified by comparing the measures in the segmentation with the
known morphological measurements of Duroc sperm (A).

(A) No CPP used

(B) Penetratin

Figure 4: Effect of penetratin. A: The reconstruction slice from a sample
stained with nanoparticles without the use of any CPP. B The reconstruction of
a slice from a sample coincubated with iron oxide nanoparticles and
Penetratin. In the sample where no CPP was used less high contrast areas could
be found compared to the sample where the CPP Penetratin was used. Without the
CPP we were only able to segment head like structures, while in the sample
where the CPP Penetratin had been used we were able to segment cylindrical
attachments to the heads consistent with the midpiece section of the cells.

To assess the success of our contrast method, we performed SXRT of fixed sperm
stained with nanoparticles and segmented the volumes. We used morphological
measurements of the sperm and compared them to our segmentations.

We used boar sperm of the Duroc breed to conduct our experiment as they are
relatively robust for storage and transportation and readily available in
large quatities.

Typical sperm, as illustrated in Figure 3(A), consists of a head, a midpiece,
and a tail. The sperm of the Duroc boar that we used have been reported to
have a flat oval head $9.4\pm 0.3\text{\,}\mathrm{\SIUnitSymbolMicro m}$ by
$4.8\pm 0.2\text{\,}\mathrm{\SIUnitSymbolMicro m}$ and a tail $43.7\pm
1.2\text{\,}\mathrm{\SIUnitSymbolMicro m}$ long [32]. To estimate the midpiece
and tail morphology, we acquired a 3D TEM image stack of a Duroc sperm pellet
and segmented it. From these segmentations, we estimate that the midpiece is
about $11\pm 0.1\text{\,}\mathrm{\SIUnitSymbolMicro m}$ long and $0.9\pm
0.1\text{\,}\mathrm{\SIUnitSymbolMicro m}$ wide. We estimate that the tail is
approximately $500\text{\,}\mathrm{nm}$ in diameter at its widest place
shortly after the neck and that the tail slowly decreases in width towards the
tip to approximately $300\text{\,}\mathrm{nm}$ in diameter. These measurements
are consistent with the sperm morphology measures on domestic pigs in [33].

We chose to conduct our experiments at the TOMCAT Nanoscope (SLS), as it has a
$75\text{\,}\mathrm{\SIUnitSymbolMicro m}$ field of view, large enough to
capture a whole cell and a $65\text{\,}\mathrm{nm}$ effective pixel size high
enough to capture the tail. The samples we tested were all fixed Epon embedded
samples as these are easier to work with than living hydrated samples. The
density of water and Epon are both close to
$1\text{\,}\mathrm{g}\text{/}\mathrm{ml}$, making the attenuation enhancement
determined by the Epon samples comparable to what we would gain in a fluid
sample.

We tested four samples at the synchrotron:

  1. 1.

Unstained cells.

  2. 2.

Cells that had been incubated with magnetite nanoparticles without the use of
Penetratin.

  3. 3.

Cells that had been incubated with magnetite nanoparticles using Penetratin to
aid the uptake.

In sample 1, Figure 3(B), we were able to identify blob-like structures
$2\text{\,}\mathrm{\SIUnitSymbolMicro m}3\text{\,}\mathrm{\SIUnitSymbolMicro
m}$ in diameter, which we believe are dense DNA inside the sperm heads. We
could not identify any structures resembling complete sperm heads or any tails
structures in this sample.

In sample 2, we were able to identify disc like structures congruent with the
flat oval shape of Duroc sperm heads, suggesting that the particles were taken
up by the sperm and thereby increase contrast.

In sample 3, we were able to see the head and midpieces of the sperm. We
detected several bright spots, consistent with enhanced contrast due to
nanoparticles, surrounded by very low contrast lighter areas which we believe
is the cell tissue. See Figure 4 for a comparison of sample 2 and 3.

The segmentation of sample 3 showed oval disc-like structures, with a
cylindrical attachment consistent with sperm heads and midpiece. The shape and
size of the disc structures are consistent with the expected shape and size of
a Duroc sperm head, and the length of the cylindrical attachment matches that
of a Duroc sperm midpiece. The results are displayed in Figure 3(B).

The results from samples 2 and 3 suggest that magnetic nanoparticles can be
used to enhance the contrast of sperm in synchrotron imaging at high energies.
By taking advantage of the membrane permeating properties of the CPP
Penetratin [26] we were able to increase the uptake of magnetic nanoparticles
and thereby the contrast.

##  3 Discussion

_Achieving dynamic SXRT of sperm cells is a compromise between three factors;
spatial resolution, temporal resolution and dose. Our results show that it is
possible to use iron oxide nanoparticles for enhancing contrast in sperm cells
during high-energy SXRT. Synchrotron imaging of live cells is extremely
difficult as it poses a multitude of sample preparation difficulties, such as
how to avoid bubbles in wet sample, how to catch a cell and store it in a
small enough container to fit withing the field of view, how to protect it
from radiation damage and how to obtain adequate contrast for analysis. While
a non-toxic staining method does not solve all of these issues it is a
necessary step towards imaging live cells in the future._

Acquiring X-ray contrast at $10\text{\,}\mathrm{keV}$, as we have done,
requires labelling of the cells. Several works have shown that it is possible
to increase contrast in X-ray projections by adding contrast enhancers such as
osmium, gadolinium, iodine or gold nanoparticles [19, 20, 21, 22, 23].
However, to the best of our knowledge, no other existing contrast method can
be used with sperm cells without compromising their motility before the
synchrotron scan. We also believe that the staining method could potentially
be used to prepare other biological samples for SXRT, where survival of the
specimen is important. The synthesis and internalization of iron oxide
nanoparticles in sperm cells has previously been demonstrated, [25], but the
novelty of our work is demonstrating that by using the CPP Penetratin, it is
possible to increase the contrast at high energy SXRT experiments enough to
visualize sperm heads and midpieces.

Increasing the energy has the potential to decrease radiation damage to the
sample. Two factors are important in lowering radiation damage; time and dose.
The absorbed dose is measured as the absorbed energy pr. unit mass and is
dependent on the X-ray mass attenuation coefficient of the sample. Higher mass
attenuation is directly related to a higher dose [34]. The mass attenuation
coefficients of soft tissue, similar in composition to sperm cells, decrease
as energy increases [35]. Hence, obtaining contrast of cells at higher
energies is an essential step towards a viable tradeoff between dose and
resolution for dynamic SXRT of live cells.

Since our future aim is to reconstruct the dynamic tail-beat of a sperm cell,
an open question is whether the particles can also provide contrast in the
tail region. As seen in the EM results in Figure 2, we found areas where
several particles had entered the cells together instead of single particles
evenly distributed. We observed particles in all regions of the cell,
including the tail.

The synchrotron images corroborate the theory that the particles tend to group
together inside the cells.The high contrast areas are larger in diameter than
a single iron oxide nanoparticle ($6\text{\,}\mathrm{nm}$) and the resolution
of the synchrotron ($65\text{\,}\mathrm{nm}$). This confirms that the
particles tend to aggregate inside the cells. . The SXRT volume with stained
cells also contains bright spots beside those attached to a head and midpiece,
implying that some particles had either not entered the cells or were attached
to the tails, which we could not segment. To determine whether the particles
provide contrast in the tail region, we would ideally need to image a single
cell with nanoparticles internalized devoid of extracellular particles.

The most significant limitation of the method presented in this study is the
uneven distribution of iron oxide nanoparticles in the cells. Therefore, it is
essential to determine how to get an even distribution of particles inside the
cells to improve the method for visualization of sperm tail region. Also, the
method is not meant as a tool for viewing the inner bio mechanics of the
cells, but as a means to gain enough contrast to reconstruct the shape of the
cell changing over time in 3D.

_It has previously been shown that iron oxide nanoparticles have no adverse
effect on sperm motility. In this article we show that iron oxide
nanoparticles can be used as a contrast agent in SXRT imaging of sperm heads
and midpieces. While the feasibility of our staining method is demonstrated on
sperm, the method likely applies to many other cells as iron-oxide
nanoparticles are widely used because of their low toxicity and
biocompatibility. Our results in terms of spatial resolution also cover most
other cells, as most animal and plant cells are
$1\text{\,}\mathrm{\SIUnitSymbolMicro m}100\text{\,}\mathrm{\SIUnitSymbolMicro
m}$ in diameter._

##  4 Methods and Material

###  4.1 Nanoparticle synthesis

We synthesized iron oxide nanoparticles by modifying a previously reported co-
precipitation method [31]. Afterwards, a PVA solution was added to the
nanoparticle suspension to incorporate a PVA surface coating [25].

Iron oxide colloids were synthesized by dissolving $2.08\text{\,}\mathrm{g}$
$\mathrm{FeCl_{2}\cdot 4H_{2}O}$ (CAS: 13478-10-9) and
$0.80\text{\,}\mathrm{g}$ $\mathrm{FeCl_{3}}$ (CAS:7705-08-0) into a
$25\text{\,}\mathrm{ml}$ $0.5\text{\,}\mathrm{\textsc{m}}$ $\mathrm{HCl}$
(CAS: 7647-01-0) under an $\mathrm{N_{2}}$ atmosphere. Next, the acidic
$\mathrm{Fe}$ salt solution was added dropwise to $25\text{\,}\mathrm{ml}$ of
an aqueous $1.08\text{\,}\mathrm{\textsc{m}}$ $\mathrm{NaOH}$ (CAS: 1310-73-2)
solution under stirring. A PVA solution was made by dissolving
$2.52\text{\,}\mathrm{g}$ PVA (CAS: 9002-89-5) in $50\text{\,}\mathrm{ml}$
deionized (DI) water under stirring for 30 minutes. The PVA solution was then
added to the iron oxide suspension and stirred for another 30 minutes. The
particles were washed 6 times with DI water, followed by precipitation by
centrifugation at 4500 rpm for 20 minutes, to remove $\mathrm{NaCl}$.
Precipitated particles were re-dispersed in DI water and sonicated for 30
seconds in between washes.

###  4.2 PXRD analysis

We collected X-ray diffraction data from the iron oxide particles, the PVA
solution, and a water background. Note that the iron oxide particles had not
been thoroughly dried before measurements, and a scattering signal from water
is therefore expected. The X-ray scattering pattern of magnetite was
calculated with CrystalDiffract®: a powder diffraction program for Mac and
Windows. CrystalMaker Software Ltd, Oxford, England (www.crystalmaker.com). We
used a transmission setup on an Empyrean Series 3 diffractometer from
Panalytical with a Scintillation Detector and a $\mathrm{AgK\alpha}$ X-ray
source. Data from the particles and PVA solution were measured in 1.0 mm
borosilicate capillaries, and data from water were measured in a 0.8 mm
borosilicate capillary. The size of the magnetite crystallites was estimated
with the Scherrer equation [36, 37], Equation 1.

$L=\frac{K\cdot\lambda}{FWHM\cdot\cos(\theta)}$ (1)

Where K is the Scherrer constant (0.9), $\lambda$ is the wavelength of X-rays
used ($0.56\text{\,}\mathrm{\text{Å}}$), FWHM is the width of the peak at half
maximum ($2\theta=0.52^{\circ}$) and $\theta$ is the angle of the chosen peak
($2\theta=12.67^{\circ}$).

###  4.3 Incubation of sperm with nanoparticles

Figure 5: Overview of the incubation procedure. The first row shows how to
disperse the nanoparticles thouroughly in the HTF buffer suitable for sperm.
The second row shows how to clean the sperm to get rid of objects otherwise
attracting the iron oxide nanoparticles. The last row shows how to prepare the
final incubation fluid and incubate the cells.

The rinsed magnetic nanoparticles were ground with a mortar to a fine powder
and $52\text{\,}\mathrm{mg}$ was added to $20\text{\,}\mathrm{ml}$
$4\text{\,}\mathrm{m\textsc{m}}$ human tubular fluid (HTF) in a
$50\text{\,}\mathrm{ml}$ falcon tube. The suspension was then whirled for a
short time at maximum power, heated to approximately
$45\text{\,}\mathrm{\SIUnitSymbolCelsius}$ and sonicated for 30 minutes. This
was repeated three times until the particles were nicely dispersed in the
liquid.

For our test, we used boar sperm of the Duroc breed. The sperm was cleaned
prior to incubation using PureSperm 50111PureSperm 50: Mix 3 parts PureSperm
40 (Nidacon, PS40-100) and 1 part PureSperm 80 (Nidacon, PS80-100). E.g
$9\text{\,}\mathrm{ml}$ PureSperm 40 and $3\text{\,}\mathrm{ml}$ PureSperm 80.
Vortex. Store in refrigirator to get rid of non-sperm objects which seem to
attract the nanoparticles. First, $1\text{\,}\mathrm{ml}$ of the sample was
placed on $2\text{\,}\mathrm{ml}$ cold PureSperm 50 in a
$15\text{\,}\mathrm{ml}$ falcon tube. Next, we spun the sample at
$400\text{\,}\mathrm{g}$ for 15 minutes at the lowest acceleration and
deceleration and carefully removed the top 2 layers and $3/4$ of layer three
such that approximately $0.5\text{\,}\mathrm{ml}$ of the cloudy fluid remained
above the pellet. We made HTF+ by adding
$150\text{\,}\mathrm{\SIUnitSymbolMicro l}$ sodium pyruvate to
$50\text{\,}\mathrm{ml}$ HTF buffer. The sperm sample was resuspended and
diluted with HTF+ until the concentration was
$13\text{\times}{10}^{6}\text{\,}{\mathrm{ml}}^{-1}$. We then made a pellet
from $1\text{\,}\mathrm{ml}$ of the rinsed sperm suspension by spinning 10
minutes at 700 rcf and removing the fluid.

Right before incubation, the nanoparticle suspension was filtered in a
$0.2\text{\,}\mathrm{\SIUnitSymbolMicro m}$ filter to get rid of any large
aggregates. After filtration we added $15\text{\,}\mathrm{\SIUnitSymbolMicro
l}$ sodium pyruvate per $5\text{\,}\mathrm{ml}$ of the filtrate and
$30\text{\,}\mathrm{\SIUnitSymbolMicro l}$ Human Serum Albumin (HSA) per
$1\text{\,}\mathrm{ml}$ of the filtrate and vortexed the suspension. Finally,
$1.5\text{\,}\mathrm{ml}$ of the buffer solution containing nanoparticles was
added to the sperm pellet and $27\text{\,}\mathrm{\SIUnitSymbolMicro l}$
Penetratin stock for a final Penetratin concentration of
$5\text{\,}\mathrm{\SIUnitSymbolMicro\textsc{m}}$.

The sperm were incubated at $37\text{\,}\mathrm{\SIUnitSymbolCelsius}$ for 80
minutes under foil and gentle. The final pellet was created by spinning at 700
rcf for 10 minutes, removing the top and slowly adding
$500\text{\,}\mathrm{\SIUnitSymbolMicro l}$ glutaraldehyde to the pellet for
fixation. See Figure 5. Due to the need to filter the nanoparticle suspension,
it is not possible to accurately report the final nanoparticle concentration.

####  4.3.1 EM imaging

Samples were fixed with $2\%$ v/v glutaraldehyde in
$0.05\text{\,}\mathrm{\textsc{m}}$ sodium phosphate buffer (pH 7.2). Following
centrifugation, the sample pellets were resuspended and rinsed in
$0.15\text{\,}\mathrm{\textsc{m}}$ sodium cacodylate buffer (pH 7.2) three
times. Next, the sample pellets were embedded in low-melting-point Agarose and
postfixed in $1\%$ w/v $\mathrm{OsO_{4}}$ in
$0.12\text{\,}\mathrm{\textsc{m}}$ sodium cacodylate buffer (pH 7.2) for
$2\text{\,}\mathrm{h}$. The specimens were dehydrated in graded series of
ethanol, transferred to propylene oxide and embedded in Epon according to
standard procedures. Sections, approximately $60\text{\,}\mathrm{nm}$ thick,
were cut with an Ultracut UCT microtome (Leica, Wienna, Austria) and collected
on copper grids with Formvar supporting membranes, stained with uranyl acetate
and lead citrate, and subsequently examined with a Philips CM 100 TEM
(Philips, Eindhoven, The Netherlands), operated at an accelerating voltage of
$80\text{\,}\mathrm{kV}$ and equipped with an OSIS Veleta digital slow-scan 2k
x 2k CCD camera (Olympus, Germany). Digital images were recorded with the ITEM
software package.

####  4.3.2 Synchrotron imaging

The samples were prepared for the Nanoscope experiment by encasing them in
EPON blocks, similar to the EM preparation but without any osmium staining.
The blocks were then cut into small rods with a Dremel saw, such that the
sample was centred in the rod. The rod was then inserted into the Dremel and
rotated at minimum or close to minimum speed while carefully sanding the
sample using sandpaper with decreasing grain sizes until the sample was below
$500\text{\,}\mathrm{\SIUnitSymbolMicro m}$ in diameter. These samples were
imaged using SXRT at the TOMCAT nanoscope at $10\text{\,}\mathrm{keV}$.

_The TOMCAT Nanoscope is a high-resolution full-field transmission X-ray
microscope based on Fresnel Zone Plate (FZP) focusing. This setup is working
in the hard X-ray regime with energies from $8\text{\,}\mathrm{keV}$ to
$20\text{\,}\mathrm{keV}$. The instrument is composed of different optical
elements: a custom designed beamshaper [38], an order selected aperture (OSA),
a fresnel zone plate (FZP) and corresponding Zernike phase rings (ZPR). The
beamshaper having a total diameter of $2975\text{\,}\mathrm{\SIUnitSymbolMicro
m}$ is made of iridium. It has an outermost zone width of
$60\text{\,}\mathrm{nm}$, a square sub-field size with side of
$75\text{\,}\mathrm{\SIUnitSymbolMicro m}$ and radial periodicity of
$75.75\text{\,}\mathrm{\SIUnitSymbolMicro m}$ between each ring of sub-fields.
It produces a top-flat illumination in its focal plane at 1684mm, at a
distance of $18.85\text{\,}\mathrm{m}$ from the X-ray source at
$10\text{\,}\mathrm{keV}$. The sample is then placed in his focal spot, having
the size of the beamshaper sub-field (i.e. $75\times 75\mu m^{2}$). The OSA is
adjusted to keep only the first order of diffraction: the direct beam is
stopped by a circular tungsten beamstop with a diameter of
$2\text{\,}\mathrm{mm}$. The Fresnel Zone Plate (FZP) acts as an objective
lens, i.e. magnifying the image on the detector. At $10\text{\,}\mathrm{keV}$,
we choose a zone plate in iridium with a diameter of
$204.6\text{\,}\mathrm{\SIUnitSymbolMicro m}$ and a focal length of
$99\text{\,}\mathrm{mm}$. In order to enhance the contrast using Zernike phase
contrast, Zernike phase rings are placed at the back-focal plane of the FZP.
This is the usual contrast mechanism used for hard X-ray imaging with TXM to
image low-Z materials such as biological tissue that usually are transparent
to hard X-rays. Projections of the sample are collected onto the detector. In
our case, the detector, consists of a Gadox scintillator screen (to convert
X-rays to visible light), optical coupling fibres, and an sCMOS sensor
(customized PCO Edge 4.2 by Rigaku, $6.5\text{\,}\mathrm{\SIUnitSymbolMicro
m}$ pixel size) situated about $10\text{\,}\mathrm{m}$ after the sample. The
X-ray magnification in such a setup is obtained by the geometrical
magnification, i.e. the ratio of the distance between the objective and the
detector (in this case $10\text{\,}\mathrm{m}$) and the distance between the
sample and the objective (in this case $100\text{\,}\mathrm{mm}$). With the
TOMCAT nanoscope a constant magnification of x100 is obtained in the energy
range $8\text{\,}\mathrm{keV}20\text{\,}\mathrm{keV}$, hence leading to an
effective pixel size of $65\text{\,}\mathrm{nm}$._

For the synchrotron samples we used an energy of $10\text{\,}\mathrm{keV}$ and
collected 1500 projections over 180 degrees, associated with two series of 50
flats and one series of 20 darks. Exposure time was set to
$150\text{\,}\mathrm{ms}$. All reconstructions were achieved using the Gridrec
reconstruction algorithm as implemented at the beamline [39]. 3D datasets
generated were saved in 16bits.

####  4.3.3 Segmentation

We segmented the cells in the reconstructed volume using scikit-image [40]. We
used a standard Gaussian filter with a kernel size of 3.0 on all axis in the
volume.

The volume was then thresholded using a triangle threshold [41]. Finally, we
found all the connected components and deleted those containing less than
80000 voxels. The threshold of 80000 voxels was determined by visual
inspection.

The visual illustrations in the article were created in BioRender.com.

##  5 Author Contributions

Conceptualization, M.B.L, A.B, T.D, K.A, J.S and M.K; Methodology, S.R.C,
A.S.A, K.A, K.Q, and M.B.L; Formal Analysis, M.B.L, S.R.C, and A.S.A;
Investigation, M.B.L, S.R.C, A.S.A, K.Q and A.B; Writing - Original Draft,
M.B.L, S.R.C, A.S.A, K.Q, A.B, and M.K; Writing - Review & Editing, A.B, T.D,
K.A, J.S, M.K, A.S.A, S.R.C, M.B.L, K.Q and K.M.Ø.J; Visualization, M.B.L,
A.S.A and S.R.C; Supervision, J.S, T.D and K.M.Ø.J; Funding Acquisition, J.S.,
T.D. and K.M.Ø.J.

##  6 Acknowledgement

MBL was supported by the Capital Region Research Foundation (Grant number:
A5657) on the MAX4Imagers project (PI: Tim Dyrby).

ASA, SRC and KMØJ are grateful to the Villum Foundation (grant number:
VKR00015416) and the Carlsberg Foundation (grant number: CF17-0976) for
funding.

We acknowledge the Paul Scherrer Institut, Villigen, Switzerland, for the
provision of synchrotron radiation beamtime (20201724) at the beamline TOMCAT
of the Swiss Light Source.

## References

  * [1] Mariam Andersson et al.  “Axon morphology is modulated by the local environment and impacts the noninvasive investigation of its structure–function relationship”  In _Proceedings of the National Academy of Sciences_ 117.52, 2020, pp. 33649–33659  DOI: 10.1073/pnas.2012533117
  * [2] Leise Borg et al.  “Muscle fibre morphology and microarchitecture in cerebral palsy patients obtained by 3D synchrotron X-ray computed tomography”  In _Computers in Biology and Medicine_ 107, 2019, pp. 265–269  DOI: 10.1016/j.compbiomed.2019.02.008
  * [3] Rajmund Mokso et al.  “Four-dimensional in vivo X-ray microscopy with projection-guided gating”  In _Scientific Reports_ 5, 2015, pp. 1–6  DOI: 10.1038/srep08727
  * [4] T. Santos Rolo, A. Ershov, T. Kamp and T. Baumbach  “In vivo X-ray cine-tomography for tracking morphological dynamics”  In _Proceedings of the National Academy of Sciences_ 111.11, 2014, pp. 3921–3926  DOI: 10.1073/pnas.1308650111
  * [5] Simon M. Walker et al.  “In Vivo Time-Resolved Microtomography Reveals the Mechanics of the Blowfly Flight Motor”  In _PLoS Biology_ 12.3, 2014  DOI: 10.1371/journal.pbio.1001823
  * [6] Rajmund Mokso et al.  “GigaFRoST: The gigabit fast readout system for tomography”  In _Journal of Synchrotron Radiation_ 24.6, 2017, pp. 1250–1259  DOI: 10.1107/S1600577517013522
  * [7] Silja Flenner et al.  “Pushing the temporal resolution in absorption and Zernike phase contrast nanotomography: Enabling fast in situ experiments”  In _Journal of Synchrotron Radiation_ 27, 2020, pp. 1339–1346  DOI: 10.1107/S1600577520007407
  * [8] Daniil Kazantsev, Enyu Guo, A.. Phillion, Philip J. Withers and Peter D. Lee  “Model-based iterative reconstruction using higher-order regularization of dynamic synchrotron data”  In _Measurement Science and Technology_ 28.9 IOP Publishing, 2017  DOI: 10.1088/1361-6501/aa7fa8
  * [9] Anna Gonzalez-Tendero et al.  “Whole heart detailed and quantitative anatomy, myofibre structure and vasculature from X-ray phase-contrast synchrotron radiation-based micro computed tomography”  In _European Heart Journal - Cardiovascular Imaging_ 18.7, 2017, pp. 732–741  DOI: 10.1093/ehjci/jew314
  * [10] Jan Willem Buurlage et al.  “Real-time reconstruction and visualisation towards dynamic feedback control during time-resolved tomography experiments at TOMCAT”  In _Scientific Reports_ 9.1, 2019, pp. 1–11  DOI: 10.1038/s41598-019-54647-4
  * [11] Aaron T. Kuan et al.  “Dense neuronal reconstruction through X-ray holographic nano-tomography”  In _Nature Neuroscience_ 23.12 Springer US, 2020, pp. 1637–1643  DOI: 10.1038/s41593-020-0704-9
  * [12] K.. Zefreh, F.. Welford and Jan Sijbers  “Investigation on the effect of exposure time on scintillator afterglow for ultra-fast tomography acquisition”  In _Journal of Instrumentation_ 11.12, 2016  DOI: 10.1088/1748-0221/11/12/C12014
  * [13] Christoph Rau et al.  “Micro- and nano-tomography at the DIAMOND beamline I13L imaging and coherence”, 2017, pp. 28  DOI: 10.1117/12.2274514
  * [14] Julie Villanova et al.  “Fast in situ 3D nanoimaging: a new tool for dynamic characterization in materials science”  In _Materials Today_ 20.7, 2017, pp. 354–359  DOI: 10.1016/j.mattod.2017.06.001
  * [15] Christian Schiffer et al.  “Direct action of endocrine disrupting chemicals on human sperm”  In _EMBO reports_ 15.7, 2014, pp. 758–765  DOI: 10.15252/embr.201438869
  * [16] A Rehfeld et al.  “EDC IMPACT: Chemical UV filters can affect human sperm function in a progesterone-like manner”  In _Endocrine Connections_ 7.1, 2018, pp. 16–25  DOI: 10.1530/EC-17-0156
  * [17] Raffaella Carzaniga, Marie Charlotte Domart, Lucy M. Collinson and Elizabeth Duke  “Cryo-soft X-ray tomography: A journey into the world of the native-state cell”  In _Protoplasma_ 251.2, 2014, pp. 449–458  DOI: 10.1007/s00709-013-0583-y
  * [18] C. Jacobsen and J. Kirz  “X-ray microscopy with synchrotron radiation”  In _Nature Structural Biology_ 5.8 SUPPL., 1998, pp. 650–653  DOI: 10.1038/1341
  * [19] Henrik S Thomsen  “Gadolinium- or iodine-based contrast media: which choice?”  In _Acta Radiologica_ 55.7, 2014  DOI: 10.1177/0284185114533537
  * [20] Julian L. Wichmann et al.  “Diagnostic accuracy of late iodine-enhancement dual-energy computed tomography for the detection of chronic myocardial infarction compared with late gadolinium-enhancement 3-T magnetic resonance imaging”  In _Investigative Radiology_ 48.12, 2013, pp. 851–856  DOI: 10.1097/RLI.0b013e31829d91a8
  * [21] Naim Aslan, Burhan Ceylan, Mümin Mehmet Koç and Fehim Findik  “Metallic nanoparticles as X-Ray computed tomography (CT) contrast agents: A review”  In _Journal of Molecular Structure_ 1219 Elsevier B.V., 2020, pp. 128599  DOI: 10.1016/j.molstruc.2020.128599
  * [22] Chia-Chi Chien et al.  “Gold nanoparticles as high-resolution X-ray imaging contrast agents for the analysis of tumor-related micro-vasculature”  In _Journal of Nanobiotechnology_ 10.1, 2012  DOI: 10.1186/1477-3155-10-10
  * [23] David P. Cormode and Zahi A. Fayad  “Nanoparticle contrast agents for CT: Their potential and the challenges that lie ahead”  In _Imaging in Medicine_ 3.3, 2011, pp. 263–266  DOI: 10.2217/iim.11.17
  * [24] Viroj Wiwanitkit, Amornpun Sereemaspun and Rojrit Rojanathanes  “Effect of gold nanoparticles on spermatozoa: the first world report”  In _Fertility and Sterility_ 91.1 Elsevier Inc., 2009, pp. e7  DOI: 10.1016/j.fertnstert.2007.08.021
  * [25] Shirly Ben David Makhluf, Riam Qasem, Sara Rubinstein, Aharon Gedanken and Haim Breitbart  “Loading magnetic nanoparticles into sperm cells does not affect their functionality”  In _Langmuir_ 22.23, 2006, pp. 9480–9482  DOI: 10.1021/la061988z
  * [26] D. Derossi, A.H. Joliot, G. Chassaing and A. Prochiantz  “The third helix of the Antennapedia homeodomain translocates through biological membranes.”  In _Journal of Biological Chemistry_ 269.14, 1994  DOI: 10.1016/S0021-9258(17)34080-2
  * [27] Pooga Margus and Ülo Langel  “Classes of Cell-Penetrating Peptides”  In _Cell-Penetrating Peptides: Methods and Protocols_ New York, NY: Springer New York, 2015, pp. 3–28  DOI: 10.1007/978-1-4939-2806-4{\\_}1
  * [28] Arwyn Jones and Edward Sayers  “Cell entry of cell penetrating peptides: Tales of tails wagging dogs”  In _Journal of controlled release : official journal of the Controlled Release Society_ 161, 2012, pp. 582–591  DOI: 10.1016/j.jconrel.2012.04.003
  * [29] S. Jones et al.  “Intracellular translocation and differential accumulation of cell-penetrating peptides in bovine spermatozoa: evaluation of efficient delivery vectors that do not compromise human sperm motility”  In _Human Reproduction_ 28.7, 2013  DOI: 10.1093/humrep/det064
  * [30] M.. Fleet  “The structure of magnetite: two annealed natural magnetites, Fe3.005O4 and Fe2.96Mg0.04O4”  In _Acta Crystallographica Section C Crystal Structure Communications_ 40.9, 1984  DOI: 10.1107/S0108270184008489
  * [31] Lucia Babes, Benoı̂t Denizot, Gisèle Tanguy, Jean Jacques Le Jeune and Pierre Jallet  “Synthesis of Iron Oxide Nanoparticles Used as MRI Contrast Agents: A Parametric Study”  In _Journal of Colloid and Interface Science_ 212.2, 1999  DOI: 10.1006/jcis.1998.6053
  * [32] Stanisław Kondracki, Maria Iwanina, Anna Wysokińska and Małgorzata Huszno  “Comparative analysis of Duroc and Pietrain boar sperm morphology”  In _Acta Veterinaria Brno_ 81.2, 2012  DOI: 10.2754/avb201281020195
  * [33] Ni-Hao Gu, Wen-Long Zhao, Gui-Shuan Wang and Fei Sun  “Comparative analysis of mammalian sperm ultrastructure reveals relationships between sperm morphology, mitochondrial functions and motility”  In _Reproductive Biology and Endocrinology_ 17.1, 2019, pp. 66  DOI: 10.1186/s12958-019-0510-y
  * [34] F.H. Attix  “Introduction To Radiological Physics and”  In _John Wiley & Sons, Inc_, 2004, pp. 607
  * [35] James Seibert and John Boone  “X-Ray Imaging Physics for Nuclear Medicine Technologists. Part 2: X-Ray Interactions and Image Formation”  In _Journal of nuclear medicine technology_ 33, 2005, pp. 3–18
  * [36] P. Scherrer  “Kolloidchemie”  Berlin, Heidelberg: Springer Berlin Heidelberg, 1912, pp. 387–409  DOI: 10.1007/978-3-662-33916-9
  * [37] J.. Langford and A… Wilson  “Scherrer after sixty years: A survey and some new results in the determination of crystallite size”  In _Journal of Applied Crystallography_ 11.2, 1978  DOI: 10.1107/S0021889878012844
  * [38] Konstantins Jefimovs, Joan Vila-Comamala, Marco Stampanoni, Burkhard Kaulich and Christian David  “Beam-shaping condenser lenses for full-field transmission X-ray microscopy”  In _Journal of Synchrotron Radiation_ 15.1, 2008, pp. 106–108  DOI: 10.1107/S0909049507047711
  * [39] Esther H.., Federica Marone and Manuel Guizar-Sicairos  “Gridrec-MS : an algorithm for multi-slice tomography”  In _Optics Letters_ 44.9, 2019, pp. 2181–2184
  * [40] Stéfan Walt et al.  “scikit-image: image processing in Python”  In _PeerJ_ 2, 2014  DOI: 10.7717/peerj.453
  * [41] G W Zack, W E Rogers and S A Latt  “Automatic measurement of sister chromatid exchange frequency.”  In _Journal of Histochemistry & Cytochemistry_ 25.7, 1977, pp. 741–753  DOI: 10.1177/25.7.70454