# LeYOLO, New Scalable and Efficient CNN Architecture for Object Detection

Lilian Hollard
Université de Reims Champagne-Ardenne
CEA, LRC DIGIT, LICIIS
Reims, France
<EMAIL_ADDRESS>
&Lucas Mohimont
Université de Reims Champagne-Ardenne
CEA, LRC DIGIT, LICIIS
Reims, France
<EMAIL_ADDRESS>
Nathalie Gaveau
Université de Reims Champagne-Ardenne
INRAE, RIBP USC 1488
Reims, France
<EMAIL_ADDRESS>
&Luiz Angelo Steffenel
Université de Reims Champagne-Ardenne
CEA, LRC DIGIT, LICIIS
Reims, France
<EMAIL_ADDRESS>
https://github.com/LilianHollard/LeYOLO

###### Abstract

Computational efficiency in deep neural networks is critical for object
detection, especially as newer models prioritize speed over efficient
computation (FLOP). This evolution has somewhat left behind embedded and
mobile-oriented AI object detection applications. In this paper, we focus on
design choices of neural network architectures for efficient object detection
computation based on FLOP and propose several optimizations to enhance the
efficiency of YOLO-based models.

Firstly, we introduce an efficient backbone scaling inspired by inverted
bottlenecks and theoretical insights from the Information Bottleneck
principle. Secondly, we present the Fast Pyramidal Architecture Network
(FPAN), designed to facilitate fast multiscale feature sharing while reducing
computational resources. Lastly, we propose a Decoupled Network-in-Network
(DNiN) detection head engineered to deliver rapid yet lightweight computations
for classification and regression tasks.

Building upon these optimizations and leveraging more efficient backbones,
this paper contributes to a new scaling paradigm for object detection and
YOLO-centric models called LeYOLO. Our contribution consistently outperforms
existing models in various resource constraints, achieving unprecedented
accuracy and flop ratio. Notably, LeYOLO-Small achieves a competitive mAP
score of 38.2% on the COCO val with just 4.5 FLOP(G), representing a 42%
reduction in computational load compared to the latest state-of-the-art
YOLOv9-Tiny model while achieving similar accuracy. Our novel model family
achieves a FLOP-to-accuracy ratio previously unattained, offering scalability
that spans from ultra-low neural network configurations (< 1 GFLOP) to
efficient yet demanding object detection setups (> 4 GFLOPs) with 25.2, 31.3,
35.2, 38.2, 39.3 and 41 mAP for 0.66, 1.47, 2.53, 4.51, 5.8 and 8.4 FLOP(G).

##  1 Introduction

In object detection, deep neural networks aim to extract a set of bounding
boxes around each object of interest from a raw input image and categorize
them into the correct class. Therefore, gathering rich spatial information is
necessary to localize precisely different objects of interest.

While deep neural networks have enabled impressive advancements, they heavily
hinge on computationally expensive resources, limiting deployment on less
powerful devices. Cloud offers an alternative to offload the execution of
powerful models but it presents inconveniences like latency, bandwidth
limitations, and security concerns [3, 80, 74]. Indeed, efficient computation,
real-time processing, and low computational latency are crucial in practical
applications such as autonomous driving, surveillance systems, medical
imaging, and intelligent agriculture. Lightweight yet robust object detectors
find their utilities in resource-constrained environments such as mobile and
edge computing devices.

Figure 1: LeYOLO (ours) with other state-of-the-art efficient object detectors
[25, 24, 14, 61, 12, 30, 64, 71, 2].

###  1.1 State-of-the-art

Deep neural networks have always been in close competition for solving
classification, regression, segmentation, and object detection problems. Many
state-of-the-art classification models [25, 54, 24, 41, 77, 43, 69, 10, 7, 10,
61, 18, 65] designed for mobile use serve as low-cost object detectors using
the SSDLite neural network [38]. On the other hand, YOLO-type models optimized
for execution speed [50, 51, 52, 2, 30, 32, 71, 73, 9, 78, 29], follows the
evolution of computing resources, forsaking embedded devices. Fortunately,
novel YOLO-based architectures embrace efficient computation, focusing on MAC
and FLOP [45, 12, 14, 76, 16, 75]. These architectures present compelling
evidence of their utility for edge AI and industrial applications, enhancing
embedded devices and enabling direct model responses.

Indeed, there is a strong interest in "YOLO" models, which are equally popular
in industry and non-computing research fields. Indeed, YOLO is used in
medicine [1, 46, 67, 60], viticulture [56, 34, 55, 48, 22], and other fields
[47, 33, 49]. Also, the ease of use and the fast execution make YOLO an
excellent choice for researchers.

The panel of object detector possibilities extends to several options: object
detection designed for speed [52, 2, 32, 9, 78, 71, 14, 51, 50, 73, 8, 30,
29], precision [53, 39, 6, 19], and the best ratio of precision (mAP) to
calculation cost (FLOP)[12, 45, 38, 16, 14, 76, 75, 40] reducing the
computational cost (FLOP) as much as possible. This last point should make us
consider separating "low-cost" and "fast" models as if they did not correlate.

Recently, we saw an interesting pattern in the focus of YOLO models, with
increasing emphasis on execution speed at the expense of computational cost
[71, 73, 9, 78]. One major drawback of this computational cost is the
necessity to consider memory access and the possibility of parallelizing
convolution operations. With the apparent evolution of GPUs or, more
generally, of computing resources, most authors have come to the unmistakable
conclusion that it is now possible to add more filters and parameters while
formulating their architectures to parallelize the operations of a neural
network as closely as possible. The phenomena increases model calculation
costs (MAC or FLOP) while maintaining or improving model execution speed
mostly thanks to GPU upgrades over the year.

The newest YOLO models [73, 8, 71], displayed as state-of-the-art in
"lightweight" models under the guise of impressive execution speed and
optimization on high-performance computing resources, still need to be
improved for edge mobile devices.

We define a lightweight model as a neural network with few FLOP, no matter the
number of parameters or execution speed. Some papers might use parameters to
prove their model’s "lightweight" capacity. However, the same number of
parameters from a filter might have very different computing needs depending
on the input spatial size, invalidating this solution. As for speed, while
this metric is interesting, it is also biased regarding the computing
resources. Therefore, we focus on FLOP to build an efficient object detector
like EfficientDet [64], and we also provide parameters and speed to give
readers a wide spectrum of comparisons (Chapter B.1).

We present LeYOLO, a conceptually simple yet efficient architecture that
embraces computationally efficient components for object detection, following
the cores of EfficientNets [62, 63], MobileNets [54, 25, 24], and similar
innovations [35, 38, 77, 72, 28, 36, 20, 26, 41, 79, 23]. We aim to introduce
a fresh architectural approach for YOLO models, prioritizing efficient
scaling. This initiative seeks to empower mobile and embedded devices with
enhanced capabilities.

We compare LeYOLO with other state-of-the-art efficient object detectors such
as YOLOs, EfficientDet and others [25, 24, 14, 61, 12, 30, 64, 71, 2]. Our
evaluation focuses on MSCOCO [37] validation mAP and FLOP ratio, highlighting
the importance of minimal computations for embedded devices. LeYOLO
demonstrates superior performance across a broad scope of neural networks,
surpassing ultra-low networks (less than 1 FLOP(G)), mid-range networks
(between 1 and 4 FLOP(G)), and even models exceeding 4 FLOP(G), as illustrated
in the Figure 1.

In all its versions, LeYOLO achieves 25.2%, 29%, 31.3%, 35.2%, 36.4%, 38.2%,
39.3% and 41% mAP on the MSCOCO validation dataset for 0.66, 1.126, 1.47,
2.53, 3.27, 4.51, 5.8 and 8.4 FLOP(G) as Table 3 shows.

Our main contributions are as follows:

  1. 1.

Lightweight: Regarding the accuracy per FLOP ratio, LeYOLO achieves the best
accuracy compared to state-of-the-art neural networks for lightweight object
detection (between 0.5 to 8 FLOP(G)).

  2. 2.

Scaling: LeYOLO offers a new opportunity to use lightweight YOLO models with
state-of-the-art scaling efficiency for industry, edge, and embedded devices.

  3. 3.

Bags of specials: Although our research focuses on the best ratio of precision
to calculation cost, we propose different alternatives by referencing all the
options we have discovered.

  4. 4.

New architecture: We suggest new computationally efficient blocks for use in
LeYOLO and provide proof through experimentation.

  5. 5.

High reproducibility: Our research priority is improving the architecture of
deep neural networks. We achieve the results using reusable training methods
from the ultralytics API, without ImageNet [31] pre-training.

##  2 Description of the LeYOLO Architecture

Figure 2: Base building block of LeYOLO (c), highlighting the distinctions
between the classical bottleneck (a), inverted bottleneck (b) [25, 54, 24],
and our proposed approach (c).

###  2.1 Block definition

####  2.1.1 Base building block

The inverted bottleneck, initially invented by MobileNetV2 [25, 54], is the
essence of many new state-of-the-art models [62, 63, 18, 65, 43, 38, 69] for
its lightweight computation and simplicity. It is complex to achieve a level
of effectiveness surpassing depthwise convolutions in terms of FLOP
computation. Pointwise convolutions address the absence of channel-wise
correlations, which are difficult to work without. Nevertheless, in our
experiments with the inverted bottleneck block, we observed that optimizing
the number of channels could reduce efficiently the computation needs,
especially at large spatial feature map sizes. Indeed, if a block’s expansion
ratio is equal to one or if, by concatenation effect, the incoming channels
$C_{in}$ are equal to the calculated number of extended layers $C_{mid}$,
there is no need to use the first pointwise convolution in our block. We
always use the residual connection even if the first pointwise does not exist,
as long as the incoming $C_{in}$ and outgoing $C_{out}$ tensor are equal as
Figure 2 and Equation $(1)$ shows. We depict the base building block of LeYOLO
in Figure 2(c), highlighting the distinctions between the classical bottleneck
(Figure 2(a)), inverted bottleneck (Figure 2(b)) [25, 54, 24], and our
proposed approach (Figure 2(c)).

We denote $\otimes$ as a convolution between two values. For
$F_{in}\in\mathbb{R}^{1,1,C_{in},C_{mid}}$,
$F_{out}\in\mathbb{R}^{1,1,C_{mid},C_{out}}$ and
$F_{mid}\in\mathbb{R}^{k,k,1,C_{mid}}$ the convolutions with $C_{in}$ as the
number of input channel, $C_{mid}$ the number of expanded channel in the
inverted bottleneck and $C_{out}$ the number of output channel :

$y=\begin{cases}F_{out}\otimes[F_{mid}\otimes(F_{in}\otimes x)]&\text{if
$C_{in}\neq C_{mid}$}\\\ F_{out}\otimes[F_{mid}\otimes(F_{in}\otimes
x)]&\text{if $C_{in}=C_{mid}$ and $F_{in}=$ True}\\\
F_{out}\otimes[F_{mid}\otimes(x)]&\text{if $C_{in}=C_{mid}$ and $F_{in}=$
False}\\\ \end{cases}$ (1)

Similar to the state-of-the-art neural network techniques for object detection
[71, 73], we consistently implement the SiLU [11] activation function
throughout our model.

####  2.1.2 Stride strategy

Figure 3: Inverse Bottleneck semantic information variations.

Several models incorporate stride within the inverted bottleneck [25, 54, 24,
75, 63, 43, 70, 44]. However, we employ a specific channel expansion strategy.

Every semantic level of information denoted as $P_{i}$ consistently has its
number of input $C_{in}P_{i}$, output $C_{out}P_{i}$, and expanded channels
$C_{mid}P_{i}$ throughout all its hidden layers. We aim to enrich the
information flow from hidden layer $h_{i}$ at the semantic information level
$P_{i}$ to the subsequent hidden layer $h_{i+1}$ by increasing the channels
$C_{in}P_{i}$ proportionately to the anticipated channel expansion from
$C_{mid}P_{i+1}$.

With the inverted bottleneck taking the following steps:

$h_{i}=F_{out}\otimes[F_{mid}\otimes(F_{in}\otimes x)]$ (2)

The semantic information inside an strided inverted bottleneck denoted as $P$
is as follows - Figure 3:

$P_{i}\rightarrow P_{i+1}\rightarrow P_{i+1}$ (3)

Therefore, the channel expansion strategy with a stride greater than one may
resemble the following form:

$C_{in}P_{i}\rightarrow C_{mid}P_{i+1}\rightarrow C_{out}P_{i+1}\quad
with\quad C_{mid}P_{i+1}>C_{mid}P_{i}$ (4)

We could simply apply a rich expansion ratio to each block, not just those
with a stride greater than one, but doing so would significantly increase the
overall network cost.

While this strategy is not mandatory for the entire model, it does make sense
at certain junctures, such as the stem, to maximize information expansion from
expensive feature map spatial sizes, particularly with an ultimate pointwise
convolution.

###  2.2 STEM

We often refer to the term "STEM" to describe the first layers, dealing
directly with the input image and low semantic information to quickly and
efficiently reduce the spatial size and excite the initial information channel
count (regularly 3 for red, green, and blue) to a higher channel count. The
main benefit is reducing the computational cost since the total cost quickly
explodes for object detection if the layers processed are too large spatially.
Only YOLOv7 [71] use no strided convolution from the first layer. The initial
P0 (640x640 - 1280x1280) spatial size is too costly.

Looking at a sample of state-of-the-art YOLOs, only YOLOv6 [32], and YOLOv8
[29] are good examples of stems with low computational resources with a total
cost of 0.32 GFLOP for both of them when scaling down the number of channels
and layers to x0.25. These two models quickly scale down to 160x160p feature
map to compensate for the cost of sliding convolutions on spatial sizes that
are too high.

To efficiently use convolutions at large feature map sizes, we use pointwise
with standard convolutions, limited to a strict low channel number throughout
the whole STEM, to go from P0 (640x640) to P2 (160x160) - Table 1.

Table 1: Stem architecture overview for LeYOLO-small. Block | Out channels | Kernel size | Stride | Activation | Output dimension
---|---|---|---|---|---
Conv+bn+act | 16 | 3 | 2 | SiLU | 320x320
Conv+bn+act | 16 | 1 | 1 | SiLU | 320x320
Depthwise+bn+act | 16 | 3 | 2 | SiLU | 160x160
Conv+bn | 16 | 1 | 1 | SiLU | 160x160
|  |  |  | Total cost | 0.3244 GFlop

###  2.3 Efficient backbone feature extractor

Historically, object detectors have been extensively supplemented for
classification [53, 19, 38], leading to observing models using classical
classification models as feature extractors.

Firstly, we use an inverted bottleneck for its unrivaled computational cost
and cost-accuracy ratio.

Finally, for the choice of the number of layers, we made an obvious
observation based on the state of the art of neural networks: there is a focus
on layer repetition on P4 or equivalent.

Even more interestingly, using a NAS-type algorithm to select the number of
layers or repetitions, we observe the same phenomenon: layers at P4 receive
more importance than others [25, 62, 63, 61, 15, 4]. We made this observation
based on a selection of YOLO [52, 2, 30, 32], as shown in Table 2 but also on
classification models using convolutions [77, 41, 58, 57, 59], self-attention
[42, 43, 44, 68, 39], and finally, object detectors like DETR [6] based on
ResNet [20], which also focuses on equivalent $P4$ middle-flow.

We cannot afford to rely so heavily on P3 like [52, 2, 32] to create a neural
network based on low computational cost. Therefore, our backbone uses the
number of repetitions shown in Table 2 (strided inverted bottleneck included).

Table 2: Layer repetition in state-of-the-art compared to our contribution. Models | P3 | P4 | P5
---|---|---|---
YOLOv3 [52] | x8 | x8 | x4
YOLOv4 [2] | x8 | x8 | x4
YOLOv5 [30] | x6 | x9 | x3
YOLOv6 [32] | x12 | x12 | x6
LeYOLO-Nano | x2 | x5 | x4

####  2.3.1 Information Bottleneck Characteristic’s

The information bottleneck principle theory from [66] highlight two critical
aspects of learning theory concerning information. Firstly, the authors
recognize that deep neural networks (DNNs) deviate from being precisely Markov
Chains [13] as $X\rightarrow\tilde{X}\rightarrow Y$ with $X$, $\tilde{X}$ and
$Y$ being the input, the minimal sufficient statistics extracted from X and
the output respectively.

Consequently, to derive $\tilde{X}$ as the minimal sufficient statistics for
extracting meaningful features to address $Y$, DNNs need to learn how to
extract features using minimal sufficient statistics, employing the most
compact architecture possible [66].

Secondly, because DNNs only process inputs from the preceding layer $h_{i-1}$,
a direct implication involves potentially losing information that subsequent
layers cannot regain (equation $(5)$).

$I(Y;X)\geq I(Y;h_{i})\geq I(Y;h_{i+j})\quad with\quad i+j\geq i$ (5)

Expensive solutions such as column-oriented networks [5, 21] with intensive
feature sharing between each block address this issue by incorporating
intensive training blocks or adding additional detection heads at crucial
points of information segmentation, as seen very recently in YOLOv9 [73]. As
achieving equity in the equation above is feasible, the theory [66] suggests
that each layer should maximize information within itself $I(Y;h_{i})$ while
minimizing inter-layer information exchange as much as possible
$I(h_{i-1};h_{i})$. Hence, rather than augmenting computational complexity in
our model like [71, 73, 21, 5], we opted to scale it more efficiently,
integrating Dangyoon’s at al. [17] inverted bottleneck theory.

Our implementation involves minimizing inter-layer information exchange in the
form of $I(h_{0};h_{n})$, with $n$ equal to the last hidden layer of the
neural network, by ensuring that the number of input/output channels never
exceeds a difference ratio of the first hidden layer through to the last. The
number of hidden layer channels should stay within a limit defined by the
input channels at P1 and the output channel at P5, with a difference ratio of
less than 6, minimizing $I(h_{i-1};h_{i})$ in the form of $I(h_{1};h_{n})$.

$C_{h_{i}}\in[C_{h_{1}};C_{h_{n}}]\quad
with\quad\frac{C_{h_{n}}}{C_{h_{1}}}\leq 6$ (6)

Conversely, the inverted bottleneck channel expansion experimentations from
Dangyonn et al. [17] show that an expansion or reduction ratio should never
exceed 6. Therefore, we maximize $I(Y;h_{i})$ with an expansion of 3 in the
whole network. Also, during a stridded inverted bottleneck with a pointwise
channels expansion strategy, the information is excited further with a total
expansion of 6, maximizing $I(Y;h_{i})$ in between $(P_{i};P_{i+1})$. However,
we excite the information further from the inverted bottleneck of $P4$ to
$P5$, maximizing $I(Y;h_{i})$ at $(P{4};P{5})$ with an expansion of 9 (+0.5
mAP compared to an expansion of 6, more information in Chapter B.3 and B.4).

The implementation of residual connections between blocks helps minimize
$I(h_{i-1};h_{i})$ by providing information from the preceding layer
$h_{i-1}$. Dense connections [16, 27] may enhance the model, but, they require
additional memory.

###  2.4 Neck

In object detection, we call neck, the part of the model that aggregates
several levels of semantic information, sharing extraction levels from more
distant layers to the first layers.

Historically, researchers have used a PANet [79] or FPN [36] to share feature
maps efficiently, enabling multiple detection levels by linking several
semantic information $P_{i}$ to the PANet and their respective outputs as
depicted in Figure 4(a).

In this paper, we are mainly focusing on two competitors: BiFPN [64] and
YOLOF’s SiSO [8]. BiFPN shares our model’s central philosophy: using layers
with low computational cost (concatenation and additions, depthwise and
pointwise convolutions). However, BiFPN requires too much semantic information
and too many blocking states (waiting for previous layers, complex graphs),
which makes it difficult to keep up with fast execution speed.

SiSO [8], on the other hand, is interesting in its approach to object
detection. Indeed, we can see that the authors of YOLOF have decided to use a
single input and output for the model neck. Compared with other proposed
solutions in YOLOF paper, we observe a significant degradation between a neck
with multiple outputs (Single-in, Multiple-out - SiMO) and a neck with a
single output (Single-in, Single-out - SiSO). We are particularly interested
in their work on the potential efficiency of a SiMO, proving the possibility
of improving the first layers of the neck of a YOLO model by optimizing the
flow of semantic information with only one rich input.

Drawing inspiration from PAN and FPNnet, we propose a Fast PANnet (FPANet)
featuring fewer convolution layers, low number of channels and more efficient
sharing of semantic information. Our approach shares similarities with the
neck concept in YOLOv8 [29]. We reduced the computational flow between $P3$
and $P5$ post-backbone and right before the head, strengthening semantic
information levels directly into $P4$. as depicted in Figure 4(b).
Additionally, we simplified the neck, minimizing locks and waiting time, given
the limited parallelization opportunities and complexities of the
architecture. Moreover, as demonstrated in the Figure 4(c), we optimized
channel numbers to reduce computation in $P3$, where the initial pointwise
step is unnecessary, as the concatenation of the bottom-up pathway from $P4$
and $P3$ information from the backbone matches the expanded channels in the
inverted bottleneck of PAN at $P3$, as
$C_{out}{P_{4}}+C_{in}{P_{3}}=C_{mid}{P_{3}}$.

By strengthening a single input ($P4$) with minimal computations from $P3$ and
$P5$ to guide neck information, we have implemented an in-between of SiMO and
MiMO approach with a significantly reduced variation of MiMO.

Figure 4: Computational distinction between classic PANet [79] and FPANet (our
model).

###  2.5 Decoupled Network in Network Head

Until YOLOv5 [50, 51, 52, 2, 30], we had single model heads for classification
and object detection. However, since YOLOv6 [32], the model head has become a
more powerful tool, separating the block into two parts: a branch for
classification and object regression. Although very efficient, this implies an
almost doubled cost, requiring convolutions for classification and detection.

We theorize that there is no need to add spatial information other than to
refine the features extracted by the backbone and neck channels-by-channels
using lightweight depthwise convolutions.

Historically, YOLO models work as a grid, proposing a classification for each
grid pixel affined by anchor boxes at each point. Anchor boxes offered several
possible detection sizes, not only pixel-wise detection.

Through YOLO’s point-by-point grid operation, we theorize that it is possible
to simplify detection heads using pointwise convolution as a sliding multi-
layer perceptron solution pixel by pixel, resembling classification
propositions for each pixel—several depthwise convolutions for spatial-only
instructions, refines spatial relationship in between two pointwise
classifying and regressing each pixel.

With an ablation study (see Chapter B.4), we prove that using only pointwise
convolutions at the head of the model yields impressive results with 33.4 mAP
at the LeYOLO-Nano@640 scale. Refining spatial information with depthwise
convolution between pointwise convolutions pushes the model to 34.3 mAP.

As figure 5 shows, we propose DNiN (Decoupled Network-in-Nework Head), a
pointwise-centered approach with two separate pointwise operations for each
network proposal: classification and regression (bounding boxes). Pointwise
operations are essential in object detection, serving as pixel-by-pixel
classifiers and regressors within a network-in-network framework. Depthwise
convolutions are split into two 3x3 convolutions to lower the overall cost
compared to a single 5x5 convolution.

We operate two separate pointwise convolutions: one dedicated to
classification and the other to regression. The distinction emerges from
differing requirements between classification and bounding box extraction. Our
proposed DNiN head expands channels to match class numbers while maintaining
spatial dimensions at level $P_{i}$. Each pixel thus represents a potential
prediction. Employing 1x1 convolutions refers to neural network origins,
notably the NiN model [35], wherein pointwise convolutions emerged as a
substitute for the conventional multi-layer perceptron.

Figure 5: In the detection head of LeYOLO, we propose DNiN (Decoupled Network-
in-Network) Head.

###  2.6 Results

####  2.6.1 Architecture scaling

For LeYOLO, we offer a variety of models inspired by the architectural base
presented above. A classic approach involves scaling the number of channels,
layers, and input image size. Traditionally, scaling emphasizes channel and
layer configurations, sometimes incorporating various scaling patterns. As the
LeYOLO model’s stem, neck, and head are optimized, increasing the image size
above 640 does not involve a drastically high FLOP. We offer eight versions of
LeYOLO, with scaling discussed in Chapters B.4.4, from 320 to 768p, from 0.66
to 8.4 FLOP(G) as presented in Table 3.

Table 3: State-of-the-art lightweight object detector. Models | Input Size | mAP | mAP50 | mAP75 | S | M | L | FLOP(G)
---|---|---|---|---|---|---|---|---
MobileNetv3-S[25] | 320 | 16.1 | - | - | - | - | - | 0.32
MobileNetv2-x0.5[54] | 320 | 16.6 | - | - | - | - | - | 0.54
MnasNet-x0.5[61] | 320 | 18.5 | - | - | - | - | - | 0.58
LeYOLO-Nano | 320 | 25.2 | 37.7 | 26.4 | 5.5 | 23.7 | 48.0 | 0.66
MobileNetv3[24] | 320 | 22 | - | - | - | - | - | 1.02
YOLOX-Nano[14] | 320 | 25.3 | - | - | - | - | - | 1.08
NanoDet | - | 23.5 | - | - | - | - | - | 1.2
LeYOLO-Small | 320 | 29 | 42.9 | 30.6 | 6.5 | 29.1 | 53.4 | 1.126
LeYOLO-Nano | 480 | 31.3 | 46.0 | 33.2 | 10.5 | 33.1 | 52.7 | 1.47
MobileNetv2[54] | 320 | 22.1 | - | - | - | - | - | 1.6
MnasNet[61] | 320 | 23 | - | - | - | - | - | 1.68
LeYOLO-Small | 480 | 35.2 | 50.5 | 37.5 | 13.3 | 38.1 | 55.7 | 2.53
Tinier-YOLO | - | 17 | 34 | 15.7 | 4.8 | 17.3 | 26.8 | 2.563
MobileNetv1[25] | 320 | 22.2 | - | - | - | - | - | 2.6
LeYOLO-Medium | 480 | 36.4 | 52.0 | 38.9 | 14.3 | 40.1 | 58.1 | 3.27
LeYOLO-Small | 640 | 38.2 | 54.1 | 41.3 | 17.6 | 42.2 | 55.1 | 4.5
YOLOv5-n[30] | 640 | 28 | 45.7 | - | - | - | - | 4.5
EfficientDet-D0[64] | 512 | 33.80 | 52.2 | 35.8 | 12 | 38.3 | 51.2 | 5
LeYOLO-Medium | 640 | 39.3 | 55.7 | 42.5 | 18.8 | 44.1 | 56.1 | 5.8
YOLOv7-Tiny[71] | 416 | 33.3 | 49.9 | - | - | - | - | 5.8
YOLOX-Tiny[14] | 416 | 32.8 | 50.3 | - | - | - | - | 6.5
YOLOv4-tiny[2] | - | 21.7 | - | - | - | - | - | 6.96
YOLOv9-Tiny[73] | 640 | 38.3 | 53.1 | 41.3 | - | - | - | 7.7
LeYOLO-Large | 768 | 41 | 57.9 | 44.3 | 21.9 | 46.1 | 56.8 | 8.4
YOLOv6-N[32] | 640 | 35.9 | 51.2 | - | - | - | - | 11.1

##  3 Discussions

As we try to offer thorough theoretical insights from state-of-the-art neural
networks to craft optimized solutions, we acknowledge several areas for
potential improvement, and we cannot wait to see further research advancements
with LeYOLO.

Channels selection: Our research still has uncertainties despite grounding our
network design in the optimal information principle theory. Observing papers
on Neural Architecture Search (NAS) reveals that the potential choices for
channel configurations are vast and often beyond human intuition. Without NAS,
it is challenging to affirm the selection of a specific number of channels
with robust theoretical evidence. We anticipate that NAS could facilitate the
discovery of better layer repetitions and channel configurations.

LeYOLO FPANet + DNiN Head: Considering the cost-effectiveness of our FPANet
and model head, there is a significant opportunity for experimentation across
different backbones of state-of-the-art classification models. LeYOLO emerges
as a promising alternative to SSD and SSDLite. The promising results achieved
on MSCoco with our solution suggest potential applicability to other
classification-oriented models We focused our optimization efforts
specifically on MSCOCO and YOLO-oriented networks. However, we encourage
experimentation with our solution on other datasets as well.

Computational efficiency: We have implemented a new scaling for YOLO models,
proving that it is possible to achieve very high levels of accuracy while
using very few computational resources (FLOP). Nevertheless, we are not the
fastest in the state of the art, as there are speed imperfections due to the
(deliberate) lack of parallelizable architecture like our predecesors [54, 24,
64]. We could further analyze scaling for different edge powers to propose
parallelizable column and block scaling.

Hence, we encourage further experimentation with our proposal, going deeper
into experimental outcomes while exploring various dataset variants tailored
to specific industry needs, such as intelligent agriculture and medicine.

##  4 Conclusion

This paper presents advancements in lightweight object detection suitable for
mobile applications, introducing a novel scaling approach for YOLO models. Our
proposed architecture, FPAN and DNiN, use lightweight operations like
pointwise and depthwise convolutions, offering a highly efficient neural
network explicitly crafted for object detection, as lightweight as SDDLite
based solutions. This breaks new ground in the cost-to-accuracy ratio.
Building these optimizations leverages more efficient YOLO scaling. We propose
a new family of YOLO models achieving competitive mAP scores on MSCOCO,
focusing on a spectrum of FLOP resource constraints. Our scaled LeYOLO-Small
achieves similar accuracy with 42% less FLOP than the newest state-of-the-art
object detectors. In this way, we demonstrate that scaling and optimized
channel selection between the different levels of semantic information can
enable us to exceed the unprecedented FLOP per mAP ratio. LeYOLO-Medium
outperforms the latest YOLOv9-Tiny state-of-the-art with 39.3 mAP (+2.61%) and
5.8 FLOP(G) (-24.67%).

## Acknowledgments and Disclosure of Funding

This work was supported by Chips Joint Undertaking (Chips JU) in EdgeAI “Edge
AI Technologies for Optimised Performance Embedded Processing” project, grant
agreement No 101097300.

## References

  * [1] Mohammed A. Al-masni, Mugahed A. Al-antari, Jeong-Min Park, Geon Gi, Tae-Yeon Kim, Patricio Rivera, Edwin Valarezo, Mun-Taek Choi, Seung-Moo Han, and Tae-Seong Kim.  Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system.  Computer Methods and Programs in Biomedicine, 157:85–94, April 2018.
  * [2] Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao.  Yolov4: Optimal speed and accuracy of object detection.  arXiv preprint arXiv:2004.10934, April 2020.
  * [3] Rajkumar Buyya, Chee Shin Yeo, Srikumar Venugopal, James Broberg, and Ivona Brandic.  Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility.  Future Generation Computer Systems, 25(6):599–616, June 2009.
  * [4] Han Cai, Ligeng Zhu, and Song Han.  Proxylessnas: Direct neural architecture search on target task and hardware.  International Conference on Learning Representations, arXiv:1812.00332, 2018.
  * [5] Yuxuan Cai, Yizhuang Zhou, Qi Han, Jianjian Sun, Xiangwen Kong, Jun Li, and Xiangyu Zhang.  Reversible column networks.  arXiv preprint arXiv:2212.11696, 2023.
  * [6] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.  End-to-End Object Detection with Transformers.  In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer Vision – ECCV 2020, pages 213–229, Cham, 2020.
  * [7] Chengpeng Chen, Zichao Guo, Haien Zeng, Pengfei Xiong, and Jian Dong.  Repghost: a hardware-efficient ghost module via re-parameterization.  arXiv preprint arXiv:2211.06088, 2022.
  * [8] Qiang Chen, Yingming Wang, Tong Yang, Xiangyu Zhang, Jian Cheng, and Jian Sun.  You only look one-level feature.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13039–13048, 2021.
  * [9] Yuming Chen, Xinbin Yuan, Ruiqi Wu, Jiabao Wang, Qibin Hou, and Ming-Ming Cheng.  Yolo-ms: rethinking multi-scale representation learning for real-time object detection.  arXiv preprint arXiv:2308.05480, 2023.
  * [10] Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, and Jian Sun.  Repvgg: Making vgg-style convnets great again.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13733–13742, 2021.
  * [11] Stefan Elfwing, Eiji Uchibe, and Kenji Doya.  Sigmoid-weighted linear units for neural network function approximation in reinforcement learning.  Neural Networks, 107:3–11, November 2018.
  * [12] Wei Fang, Lin Wang, and Peiming Ren.  Tinier-YOLO: A Real-Time Object Detection Method for Constrained Environments.  IEEE Access, 8:1935–1944, 2020.
  * [13] Paul A. Gagniuc.  Markov Chains: From Theory To Implementation And Experimentation.  John Wiley and Sons, Inc, 2017.
  * [14] Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, and Jian Sun.  Yolox: Exceeding yolo series in 2021.  arXiv preprint arXiv:2107.08430, August 2021.
  * [15] Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le.  Nas-fpn: Learning scalable feature pyramid architecture for object detection.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7036–7045, 2019.
  * [16] Mohammad Hajizadeh, Mohammad Sabokrou, and Adel Rahmani.  MobileDenseNet: A new approach to object detection on mobile devices.  Expert Systems with Applications, 215:119348, April 2023.
  * [17] Dongyoon Han, Sangdoo Yun, Byeongho Heo, and YoungJoon Yoo.  Rethinking channel dimensions for efficient model design.  In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 732–741, 2021.
  * [18] Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu.  Ghostnet: More features from cheap operations.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1580–1589, 2020.
  * [19] Kaiming He, Georgia Gkioxari, Piotr Dollar, and Ross Girshick.  Mask r-cnn.  In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2961–2969, Oct 2017.
  * [20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [21] Geoffrey Hinton.  How to Represent Part-Whole Hierarchies in a Neural Network.  Neural Computation, 35(3):413–452, February 2023.
  * [22] Lilian Hollard and Lucas Mohimont.  Applying Knowledge Distillation on Pre-Trained Model for Early Grapevine Detection.  In Workshop Proceedings of the 19th International Conference on Intelligent Environments (IE2023), pages 149–156. IOS Press, 2023.
  * [23] Lilian Hollard, Lucas Mohimont, and Luiz Angelo Steffenel.  Designing Lightweight CNN for Images: Architectural Components and Techniques.  In Advancing Edge Artificial Intelligence. River Publishers, 2024.
  * [24] Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al.  Searching for mobilenetv3.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 1314–1324, 2019.
  * [25] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam.  Mobilenets: Efficient convolutional neural networks for mobile vision applications.  arXiv preprint arXiv:1704.04861, 2017.
  * [26] Jie Hu, Li Shen, and Gang Sun.  Squeeze-and-excitation networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132–7141, 2018.
  * [27] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.  Densely connected convolutional networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.
  * [28] Forrest N Iandola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer.  Squeezenet: Alexnet-level accuracy with 50x fewer parameters and< 0.5 mb model size.  arXiv preprint arXiv:1602.07360, 2016.
  * [29] Glenn Jocher, Ayush Chaurasia, and Jing Qiu.  Ultralytics YOLO, January 2023.
  * [30] Glenn Jocher, Ayush Chaurasia, Alex Stoken, Jirka Borovec, NanoCode012, Yonghye Kwon, Kalen Michael, TaoXie, Jiacong Fang, imyhxy, Lorna, (Zeng Yifu), Colin Wong, Abhiram V, Diego Montes, Zhiqiang Wang, Cristi Fati, Jebastin Nadar, Laughing, UnglvKitDe, Victor Sonck, tkianai, yxNONG, Piotr Skalski, Adam Hogan, Dhruv Nair, Max Strobel, and Mrinal Jain.  ultralytics/yolov5: v7.0 - YOLOv5 SOTA Realtime Instance Segmentation, November 2022.
  * [31] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  ImageNet Classification with Deep Convolutional Neural Networks.  In Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012.
  * [32] Chuyi Li, Lulu Li, Hongliang Jiang, Kaiheng Weng, Yifei Geng, Liang Li, Zaidan Ke, Qingyuan Li, Meng Cheng, Weiqiang Nie, et al.  Yolov6: A single-stage object detection framework for industrial applications.  arXiv preprint arXiv:2209.02976, September 2022.
  * [33] Gang Li, Shilong Zhao, Mingle Zhou, Min Li, Rui Shao, Zekai Zhang, and Delong Han.  Yolo-rff: An industrial defect detection method based on expanded field of feeling and feature fusion.  Electronics, 11(24):4211, 2022.
  * [34] Huipeng Li, Changyong Li, Guibin Li, and Lixin Chen.  A real-time table grape detection method based on improved YOLOv4-tiny network in complex background.  Biosystems Engineering, 212:347–359, December 2021.
  * [35] Min Lin, Qiang Chen, and Shuicheng Yan.  Network in network.  arXiv preprint arXiv:1312.4400, 2013.
  * [36] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie.  Feature pyramid networks for object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117–2125, 2017.
  * [37] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014.
  * [38] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C. Berg.  Ssd: Single shot multibox detector.  In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision – ECCV 2016, pages 21–37, Cham, 2016.
  * [39] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021.
  * [40] Yonghui Lu, Langwen Zhang, and Wei Xie.  YOLO-compact: An Efficient YOLO Network for Single Category Real-time Object Detection.  In 2020 Chinese Control And Decision Conference (CCDC), pages 1931–1936, August 2020.
  * [41] Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun.  Shufflenet v2: Practical guidelines for efficient cnn architecture design.  In Proceedings of the European conference on computer vision (ECCV), pages 116–131, 2018.
  * [42] Muhammad Maaz, Abdelrahman Shaker, Hisham Cholakkal, Salman Khan, Syed Waqas Zamir, Rao Muhammad Anwer, and Fahad Shahbaz Khan.  EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications.  In Leonid Karlinsky, Tomer Michaeli, and Ko Nishino, editors, Computer Vision – ECCV 2022 Workshops, Lecture Notes in Computer Science, pages 3–20, Cham, 2023.
  * [43] Sachin Mehta and Mohammad Rastegari.  Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer.  In International Conference on Learning Representations, 2022.
  * [44] Sachin Mehta and Mohammad Rastegari.  Separable Self-attention for Mobile Vision Transformers, June 2022.
  * [45] Julian Moosmann, Marco Giordano, Christian Vogt, and Michele Magno.  Tinyissimoyolo: A quantized, low-memory footprint, tinyml object detection network for low power microcontrollers.  In 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS), pages 1–5, 2023.
  * [46] Yali Nie, Paolo Sommella, Mattias O’Nils, Consolatina Liguori, and Jan Lundgren.  Automatic detection of melanoma with yolo deep convolutional neural networks.  In 2019 E-Health and Bioengineering Conference (EHB), pages 1–4. IEEE, 2019.
  * [47] Dinh-Lam Pham, Tai-Woo Chang, et al.  A yolo-based real-time packaging defect detection system.  Procedia Computer Science, 217:886–894, 2023.
  * [48] Isabel Pinheiro, Germano Moreira, Daniel Queirós da Silva, Sandro Magalhães, António Valente, Paulo Moura Oliveira, Mário Cunha, and Filipe Santos.  Deep learning yolo-based solution for grape bunch detection and assessment of biophysical lesions.  Agronomy, 13(4):1120, 2023.
  * [49] Nitin Rane.  Yolo and faster r-cnn object detection for smart industry 4.0 and industry 5.0: applications, challenges, and opportunities.  Available at SSRN 4624206, 2023.
  * [50] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi.  You only look once: Unified, real-time object detection.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 779–788, June 2016.
  * [51] Joseph Redmon and Ali Farhadi.  Yolo9000: Better, faster, stronger.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
  * [52] Joseph Redmon and Ali Farhadi.  Yolov3: An incremental improvement.  arXiv preprint arXiv:1804.02767, 2018.
  * [53] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  Advances in neural information processing systems, 28, 2015.
  * [54] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.  Mobilenetv2: Inverted residuals and linear bottlenecks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510–4520, 2018.
  * [55] Thiago T. Santos, Leonardo L. de Souza, Andreza A. dos Santos, and Sandra Avila.  Grape detection, segmentation, and tracking using deep neural networks and three-dimensional association.  Computers and Electronics in Agriculture, 170:105247, March 2020.
  * [56] Marco Sozzi, Silvia Cantalamessa, Alessia Cogato, Ahmed Kayad, and Francesco Marinello.  Automatic Bunch Detection in White Grape Varieties Using YOLOv3, YOLOv4, and YOLOv5 Deep Learning Algorithms.  Agronomy, 12(2):319, 2022.
  * [57] Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander Alemi.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning.  Proceedings of the AAAI Conference on Artificial Intelligence, 31(1), February 2017.
  * [58] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich.  Going Deeper With Convolutions.  pages 1–9, 2015.
  * [59] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.  Rethinking the inception architecture for computer vision.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016.
  * [60] Lu Tan, Tianran Huangfu, Liyao Wu, and Wenying Chen.  Comparison of RetinaNet, SSD, and YOLO v3 for real-time pill identification.  BMC Medical Informatics and Decision Making, 21(1):324, November 2021.
  * [61] Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V. Le.  MnasNet: Platform-Aware Neural Architecture Search for Mobile.  In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2815–2823, Long Beach, CA, USA, June 2019.
  * [62] Mingxing Tan and Quoc Le.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks.  In Proceedings of the 36th International Conference on Machine Learning, pages 6105–6114. PMLR, May 2019.
  * [63] Mingxing Tan and Quoc Le.  EfficientNetV2: Smaller Models and Faster Training.  In Proceedings of the 38th International Conference on Machine Learning, pages 10096–10106. PMLR, July 2021.
  * [64] Mingxing Tan, Ruoming Pang, and Quoc V Le.  Efficientdet: Scalable and efficient object detection.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10781–10790, 2020.
  * [65] Yehui Tang, Kai Han, Jianyuan Guo, Chang Xu, Chao Xu, and Yunhe Wang.  GhostNetV2: Enhance Cheap Operation with Long-Range Attention.  Advances in Neural Information Processing Systems, 35:9969–9982, December 2022.
  * [66] Naftali Tishby and Noga Zaslavsky.  Deep learning and the information bottleneck principle.  In 2015 IEEE Information Theory Workshop (ITW), pages 1–5, April 2015.
  * [67] Halil Murat Ünver and Enes Ayan.  Skin lesion segmentation in dermoscopic images with combination of yolo and grabcut algorithm.  Diagnostics, 9(3):72, 2019.
  * [68] Pavan Kumar Anasosalu Vasu, James Gabriel, Jeff Zhu, Oncel Tuzel, and Anurag Ranjan.  Fastvit: A fast hybrid vision transformer using structural reparameterization.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5785–5795, 2023.
  * [69] Pavan Kumar Anasosalu Vasu, James Gabriel, Jeff Zhu, Oncel Tuzel, and Anurag Ranjan.  Mobileone: An improved one millisecond mobile backbone.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7907–7917, 2023.
  * [70] Shakti N Wadekar and Abhishek Chaurasia.  Mobilevitv3: Mobile-friendly vision transformer with simple and effective fusion of local, global and input features.  arXiv preprint arXiv:2209.15159, October 2022.
  * [71] Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao.  Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7464–7475, June 2023.
  * [72] Chien-Yao Wang, Hong-Yuan Mark Liao, Yueh-Hua Wu, Ping-Yang Chen, Jun-Wei Hsieh, and I-Hau Yeh.  Cspnet: A new backbone that can enhance learning capability of cnn.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 390–391, 2020.
  * [73] Chien-Yao Wang, I-Hau Yeh, and Hong-Yuan Mark Liao.  Yolov9: Learning what you want to learn using programmable gradient information.  arXiv preprint arXiv:2402.13616, 2024.
  * [74] Fangxin Wang, Miao Zhang, Xiangxiang Wang, Xiaoqiang Ma, and Jiangchuan Liu.  Deep Learning for Edge Computing Applications: A State-of-the-Art Survey.  IEEE Access, 8:58322–58336, 2020.
  * [75] Zixuan Wang, Jiacheng Zhang, Zhicheng Zhao, and Fei Su.  Efficient Yolo: A Lightweight Model For Embedded Deep Learning Object Detection.  In 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pages 1–6, July 2020.
  * [76] Wang Yang, Ding BO, and Li Su Tong.  TS-YOLO:An efficient YOLO Network for Multi-scale Object Detection.  In 2022 IEEE 6th Information Technology and Mechatronics Engineering Conference (ITOEC), volume 6, pages 656–660, March 2022.
  * [77] Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun.  Shufflenet: An extremely efficient convolutional neural network for mobile devices.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6848–6856, 2018.
  * [78] Zixiao Zhang, Xiaoqiang Lu, Guojin Cao, Yuting Yang, Licheng Jiao, and Fang Liu.  ViT-YOLO:Transformer-Based YOLO for Object Detection.  In 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 2799–2808, October 2021.
  * [79] Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia.  Pyramid scene parsing network.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2881–2890, 2017.
  * [80] Zhi Zhou, Xu Chen, En Li, Liekang Zeng, Ke Luo, and Junshan Zhang.  Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing.  Proceedings of the IEEE, 107(8):1738–1762, August 2019.

##  Appendix A Appendix / supplemental material

##  Appendix B Notations

Throughout the document, we use several notations to describe the essential
components of deep learning, particularly in object detection—for example, the
spatial size of different tensors described as $P_{i}$. This chapter covers
all the notations used in the paper. Since there is little to no consensus on
Deep Learning notations, we found it relevant to describe them in more depth
in the appendix for readers who need further explanation.

To start, we want to provide more explanation on the main component of this
paper: the computation formula. Throughout the paper, we consistently use the
FLOP (Floating Point Operations) metric to compare our work with other state-
of-the-art neural networks. By basing computations on the number of
multiplications and additions required for the neural network, we establish a
solid foundation for efficient model comparisons. The FLOP metric remains
reliable regardless of the hardware used, making it a good indicator of
computational efficiency. Although we could use other metrics such as speed,
these are highly dependent on factors like neural network architecture
parallelization, the hardware used, the accelerator software (TensorRT,
CoreML, TFLite), and memory usage and transfer speeds.

The second main element we consistently use throughout the paper is mAP (mean
Average Precision). Researchers widely use this metric to compare object
detection-based neural networks. mAP measures the precision of the model by
evaluating the overlap between the proposed bounding boxes and the actual
annotated bounding boxes. While some papers compare models using mAP at a
fixed threshold of 50% overlap (mAP50), we primarily use mAP50-95. This metric
averages the precision over different overlap thresholds, ranging from 50% to
95%, covering a broader range of evaluation criteria.

In object detection, where the spatial size of the feature map is essential,
we define $\boldsymbol{P_{i}}$ as the feature map size of our deep neural
network. The sizes range from $P0$ (640x640 pixels) to $P5$ (20x20 pixels) for
LeYOLO-Small to Medium, with $i$ representing the number of strides used.
Similarly, $P_{i-1}$ denotes the size of the preceding feature map for the
explicitly described feature map $i$.

Similarly, when describing hidden layers in the neural network, we refer to
the entire block rather than a single convolution. For example, the paper
describes a single inverted bottleneck as one hidden layer $h_{i}$ that
consists of two pointwise convolutions and one depthwise convolution.
Throughout the paper, this lets us directly refer to the preceding inverted
bottleneck as $h_{i-1}$.

Note that we might combine all notations to highlight specific components of
the architecture. For instance, we denote the number of channels $C$ at each
semantic level $P$ as $C_{P_{i}}$. We specify the height and width of the
feature map for each semantic level $P_{i}$ as $H_{P_{i}}$ and $W_{P_{i}}$.

###  B.1 Complete State-of-the-art

This section provides a comprehensive comparison of the state-of-the-art,
juxtaposing LeYOLO with YOLO mainline models, micro neural networks designed
for object detection, and the leading classification model, SSDLite, operating
at 320x320 resolution.

We evaluate our performance against others using two primary metrics: Mean
Average Precision (mAP) and FLOPs. The mAP is computed with various
parameters, including an IOU of 0.5. For FLOP however, the intertwined nature
of MAC and FLOP formulas has resulted in inconsistencies, with researchers
erroneously labeling their models using FLOP instead of MAC. This implies that
the computational cost of the model is at least twice the stated initial
amount.

We also added the number of parameters used in a lightweight state-of-the-art
model for object detection. Except for impressive results from YOLOv9-Tiny
with two million parameters, our contributions use very few parameters
compared to others. Table 4 shows all results.

Table 4: State-of-the-art lightweight object detector. Models | Input Size | mAP | mAP50 | mAP75 | S | M | L | FLOP(G) | Parameters (M)
---|---|---|---|---|---|---|---|---|---
MobileNetv3-S[25] | 320 | 16.1 | - | - | - | - | - | 0.32 | 1.77
MobileNetv2-x0.5[54] | 320 | 16.6 | - | - | - | - | - | 0.54 | 1.54
MnasNet-x0.5[61] | 320 | 18.5 | - | - | - | - | - | 0.58 | 1.68
LeYOLO-Nano | 320 | 25.2 | 37.7 | 26.4 | 5.5 | 23.7 | 48.0 | 0.66 | 1.1
MobileNetv3[24] | 320 | 22 | - | - | - | - | - | 1.02 | 3.22
YOLOX-Nano[14] | 320 | 25.3 | - | - | - | - | - | 1.08 | 0.91
NanoDet |  | 23.5 | - | - | - | - | - | 1.2 | 0.95
LeYOLO-Small | 320 | 29 | 42.9 | 30.6 | 6.5 | 29.1 | 53.4 | 1.126 | 1.9
LeYOLO-Nano | 480 | 31.3 | 46 | 33.2 | 10.5 | 33.1 | 52.7 | 1.47 | 1.1
MobileNetv2[54] | 320 | 22.1 | - | - | - | - | - | 1.6 | 4.3
MnasNet[61] | 320 | 23 | - | - | - | - | - | 1.68 | 4.8
LeYOLO-Small | 480 | 35.2 | 50.5 | 37.5 | 13.3 | 38.1 | 55.7 | 2.53 | 1.9
Tinier-YOLO |  | 17 | 34 | 15.7 | 4.8 | 17.3 | 26.8 | 2.563 | -
MobileNetv1[25] | 320 | 22.2 | - | - | - | - | - | 2.6 | 5.1
LeYOLO-Medium | 480 | 36.4 | 52.0 | 38.9 | 14.3 | 40.1 | 58.1 | 3.27 | 2.4
LeYOLO-Small | 640 | 38.2 | 54.1 | 41.3 | 17.6 | 42.2 | 55.1 | 4.5 | 1.9
YOLOv5-n[30] | 640 | 28 | 45.7 | - | - | - | - | 4.5 | 1.9
EfficientDet-D0[64] | 512 | 33.80 | 52.2 | 35.8 | 12 | 38.3 | 51.2 | 5 | 3.9
LeYOLO-Medium | 640 | 39.3 | 55.7 | 42.5 | 18.8 | 44.1 | 56.1 | 5.8 | 1.9
YOLOv7-Tiny[71] | 416 | 33.3 | 49.9 | - | - | - | - | 5.8 | 6.2
YOLOX-Tiny[14] | 416 | 32.8 | 50.3 | - | - | - | - | 6.5 | 5.06
YOLOv4-tiny[2] | - | 21.7 | - | - | - | - | - | 6.96 | 6.06
YOLOv9-Tiny[73] | 640 | 38.3 | 53.1 | 41.3 | - | - | - | 7.7 | 2
LeYOLO-Large | 768 | 41 | 57.9 | 44.3 | 21.9 | 46.1 | 56.8 | 8.4 | 2.4
YOLOv6-N[32] | 640 | 35.9 | 51.2 | - | - | - | - | 11.1 | 4.3

###  B.2 Overall Architecture

For the sake of readability, we have omitted a discussion of the semantic
sharing strategy in the neck. We employ lightweight blocks for upsampling the
feature map in the bottom-up pathway, while the top-down pathway utilizes
standard convolutions. Despite their pronounced cost, strided standard
convolutions prove efficient in this context due to the low spatial size and
the restrained number of channels used.

We experimented with inverted bottlenecks instead of convolutions but found
them more costly and less accurate. Additionally, we refrain from using any
convolutions before the first 20x20 upsampling, diverging from the approach
taken in most YOLO architectures. We question the necessity of another
convolution after the Spatial Pyramid Pooling Fusion (SPPF) [30, 29], as it
may already be sufficiently efficient in the backbone. Lastly, the 80x80
aspect is resource-intensive and requires careful consideration. Following
similar reasoning, we avoided computation for the 80x80 top-down pathway, as
the cost seemed disproportionate to the marginal accuracy improvement, as
demonstrated in ablation studies on FPANet’s 80x80 pointwise component. Figure
6 represents the complete architecture of LeYOLO.

Figure 6: We present a visual decomposition of the LeYOLO network’s
comprehensive architecture, providing a detailed explanation of its backbone,
neck, and head components.

###  B.3 Architecture differences

We propose a comparison between our proposition and several inverted
bottleneck-inspired backbones. Despite being disregarded by most contemporary
state-of-the-art object detectors, MobileNetv2 [54], MobileNetv3 [24], and
EfficientNets [75, 63, 64] share the philosophy of employing inverted
bottlenecks for object detection.

As mentioned in the introduction, advancements in GPUs have enabled the
development of powerful and rapid neural networks. However, inverted
bottlenecks provide limited depth for parallelizing multiple computation
blocks. Parallelizing deep neural networks on embedded devices remains
challenging, but there is optimism for the future. Research primarily focuses
on reducing MAC and FLOP costs and occasionally even memory access costs.
Naturally, execution speed remains a significant concern. Nevertheless, we aim
to briefly compare our backbone (Table 5) using consistent notation with
models that exhibit "similar" architecture (Tables 6, 7 and 8), particularly
those utilizing inverted bottlenecks.

Through this comparison, we can observe the stride inverted bottleneck
strategy. Upon code verification, we indeed notice a contrast in channel
expansion when transitioning from one layer $(h_{i};P_{i})$ to another
$(h_{i+1};P_{i+1})$ with a stride greater than one. Additionally, most
inverted bottlenecks utilize an expansion ratio of 6, whereas we only expand
to 3 within a block. This reduces overall computation and allows the inverted
bottleneck stride strategy to expand the number of channels one last time
before the stride within the depthwise convolution.

Table 5: LeYOLO backbone (base scale : nano). Input | Operator | exp size | out size | NL | s
---|---|---|---|---|---
P0 | conv2d, 3x3 | - | 16 | SI | 2
P1 | conv2d, 1x1 | 16 | 16 | SI | 1
P1 | bneck, 3x3, pw=False | 16 | 16 | SI | 2
P2 | bneck, 3x3 | 96 | 32 | SI | 2
P3 | bneck, 3x3 | 96 | 32 | SI | 1
P3 | bneck, 5x5 | 96 | 64 | SI | 2
P4 | bneck, 5x5 | 192 | 64 | SI | 1
P4 | bneck, 5x5 | 192 | 64 | SI | 1
P4 | bneck, 5x5 | 192 | 64 | SI | 1
P4 | bneck, 5x5 | 192 | 64 | SI | 1
P4 | bneck, 5x5 | 576 | 96 | SI | 2
P5 | bneck, 5x5 | 576 | 96 | SI | 1
P5 | bneck, 5x5 | 576 | 96 | SI | 1
P5 | bneck, 5x5 | 576 | 96 | SI | 1
P5 | bneck, 5x5 | 576 | 96 | SI | 1
Table 6: MobileNetv2 backbone. Input | Operator | exp size | out size | NL | s
---|---|---|---|---|---
P0 | conv2d, 3x3 | - | 32 | RE | 2
P1 | bneck, 3x3 | 16 | 16 | RE | 1
P1 | bneck, 3x3 | 96 | 24 | RE | 2
P2 | bneck, 3x3 | 144 | 24 | RE | 1
P2 | bneck, 3x3 | 144 | 32 | RE | 2
P3 | bneck, 3x3 | 192 | 32 | RE | 1
P3 | bneck, 3x3 | 192 | 32 | RE | 1
P3 | bneck, 3x3 | 192 | 64 | RE | 2
P4 | bneck, 3x3 | 384 | 64 | RE | 1
P4 | bneck, 3x3 | 384 | 64 | RE | 1
P4 | bneck, 3x3 | 384 | 64 | RE | 1
P4 | bneck, 3x3 | 384 | 96 | RE | 1
P4 | bneck, 3x3 | 576 | 96 | RE | 1
P4 | bneck, 3x3 | 576 | 96 | RE | 1
P4 | bneck, 3x3 | 576 | 160 | RE | 2
P5 | bneck, 3x3 | 960 | 160 | RE | 1
P5 | bneck, 3x3 | 960 | 320 | RE | 1
P5 | conv, 1x1 | 1920 | 320 | RE | 1
Table 7: MobileNetv3 backbone. Input | Operator | exp size | out size | NL | s
---|---|---|---|---|---
P0 | conv2d, 3x3 | - | 16 | HS | 2
P1 | conv2d, 1x1 | 16 | 16 | RE | 2
P2 | bneck, 3x3 | 72 | 24 | RE | 2
P3 | bneck, 3x3 | 88 | 24 | RE | 1
P3 | bneck, 5x5 | 96 | 40 | HS | 2
P4 | bneck, 5x5 | 96 | 40 | HS | 1
P4 | bneck, 5x5 | 240 | 40 | HS | 1
P4 | bneck, 5x5 | 240 | 40 | HS | 1
P4 | bneck, 5x5 | 120 | 48 | HS | 1
P4 | bneck, 5x5 | 144 | 48 | HS | 1
P4 | bneck, 5x5 | 288 | 96 | HS | 2
P5 | bneck, 5x5 | 576 | 96 | HS | 1
P5 | bneck, 5x5 | 576 | 96 | HS | 1
Table 8: EfficientNet-B0 (EfficientDet-D0 and D1) backbone. Input | Operator | exp size | out size | NL | s
---|---|---|---|---|---
P0 | conv2d, 3x3 | - | 32 | RE | 2
P1 | bneck, 3x3 | 16 | 16 | RE | 1
P1 | bneck, 3x3 | 96 | 24 | RE | 2
P2 | bneck, 3x3 | 144 | 24 | RE | 1
P2 | bneck, 5x5 | 144 | 40 | RE | 2
P3 | bneck, 5x5 | 240 | 40 | RE | 1
P3 | bneck, 3x3 | 240 | 80 | RE | 1
P3 | bneck, 3x3 | 480 | 80 | RE | 1
P3 | bneck, 3x3 | 480 | 80 | RE | 1
P3 | bneck, 5x5 | 480 | 112 | RE | 2
P4 | bneck, 5x5 | 672 | 112 | RE | 1
P4 | bneck, 5x5 | 672 | 112 | RE | 1
P4 | bneck, 5x5 | 672 | 192 | RE | 2
P5 | bneck, 5x5 | 1152 | 192 | RE | 1
P5 | bneck, 5x5 | 1152 | 192 | RE | 1
P5 | bneck, 5x5 | 1152 | 192 | RE | 1
P5 | bneck, 3x3 | 1280 | 320 | RE | 1

###  B.4 Ablation study

It is difficult to mathematically prove universal truths, such as whether one
neural network architecture performs better than another without training.
However, we do not see this as a problem; on the contrary, we use ablation
studies to prove that our architecture is functional. Moreover, we have
encountered architectures that perform better than our final proposed
contribution but require higher computational costs. This ablation study
represents the most relevant experiments we have been able to carry out.
However, we have left it to the reader to choose the most efficient, albeit
more costly, to use them in future research to guide researchers more
effectively.

####  B.4.1 Kernel size

Kernel filter size: For narrative clarity, we directly outline the final
architecture selected for optimal accuracy and FLOP efficiency in the paper.
However, our study began with a minimalist architecture based on an inverted
bottleneck. Initially, we experimented with kernel sizes of 5x5 and 7x7 when
the feature map size reached 80x80. Our ablation study revealed that a larger
kernel size yields promising results, as indicated in Table 9. Although the
7x7 kernel outperforms 3x3 (+2,4 mAP points) and 5x5 (+0.4 mAP points), it
significantly increases FLOP requirements. Therefore, we opted for the 5x5
kernel to balance FLOP utilization and accuracy.

Kernel size ablation study:

  1. 1.

minimal: minimal architecture with classic inverted bottleneck and full 3x3
convolutions

  2. 2.

5x5 : minimal architecture with 5x5 convolutions at and after P3.

  3. 3.

7x7 : minimal architecture with 7x7 convolutions at and after P3.

With 34.9 mAP on MSCOCO and 3,293 FLOP(G), the best choice was to use a
consistent 5x5 kernel size throughout the model from semantic level P3 to P5
(Stem remains with a 3x3 kernel size).

Kernel size and feature map size cost: The P3 (80x80) feature map incurs
considerable computational costs, necessitating a reduction in computation at
this model stage. One objective is optimizing model efficiency by exclusively
employing 5x5 convolutions from P4 to P5 instead of P3 to P5 along the
backbone and FPANet.

Moreover, the effectiveness of employing two convolutions at 5x5 with a stride
of two in the top-down pathway of the FPANet still needs to be proven. We
explored the efficacy of different kernel sizes by employing 3x3 convolutions
for downsampling in the FPANet. Our ablation study identified the optimal
compromise for kernel size at this juncture, as illustrated in Table 9.

Kernel size ablation study:

  1. 1.

5x5@P4 : minimal architecture with 5x5 convolutions at and after P4.

  2. 2.

FPANet 3x3 $\downarrow$ : minimal architecture with 5x5@P4 and only 3x3
convolutions for downsampling in FPANet

Regarding the mAP to FLOP ratio, we observed a better kernel filter size
choice by combining the use of 5x5 kernel size from P4 to P5 (with Stem and P3
using a consistent 3x3 kernel size) and the use of 3x3 kernel size for FPANet
downsampling convolutions. These choices pushed the mAP to FLOP ratio to
$\frac{mAP}{FLOP(G)}=11,49$ with 34,6 mAP for 3,011 FLOP(G)

####  B.4.2 Architecture improvement

To improve the architecture, we aimed to simplify all computations before P4.
Having already made adjustments regarding kernel size at P3, we removed the
initial pointwise operation in the STEM at P2 (160x160), as the input channel
matched the number of required channels for the depthwise convolution.
Leveraging the efficiency of our inverted bottleneck building block, coupled
with optional pointwise operations, unexpectedly improved the model with 34,7
mAP and reduced computation by 66MFLOP (2,945 FLOP(G)).

In line with this optimization process, we opted for a channel configuration
in both the minimal and final architectures, aligned with a compressed
information bottleneck $I(h_{1};h_{n})$ (Equation (6)), limiting input
channels at each block. In this manner, the feature map concatenation from the
bottom-up pathway of P4 $C_{out}$ and P3 $C_{out}$, retained in memory from
the backbone, already matched the desired expanded channel number for the P3
block $C_{mid}$ in the FPANet. Thus, we streamlined the architecture to its
essential components before P4, freeing up computation resources for an
impressive 34.1 mAP for 2,823 FLOP(G).

Table 9 describes all results.

Minimal architecture improvement:

  1. 1.

$no_{pw}@STEM$ : no pointwise in the stem in the first inverted bottleneck at
P1.

  2. 2.

$no_{pw}@FPAN-P3$ : no pointwise in the first inverted bottleneck at P3 in the
bottom-up FPAN pathway.

While some studies may advocate retaining the previous configuration, arguing
that removing pointwise convolutions compromises accuracy, we pursued
alternative experiments to improve the accuracy to FLOP ratio. Despite the
potential controversy, we maintain the integrity of our ablation study,
preserving all findings for the benefit of researchers or readers who may find
value in various aspects of our experimentation.

Table 9: Ablation study on minimal inverted bottleneck architecture. minimal | 5x5 | 7x7 | 5x5@P4 |  FPANet 3x3 $\downarrow$ | $no_{pw}@STEM$ | $no_{pw}@FPAN-P3$ | mAP | FLOP(G)
---|---|---|---|---|---|---|---|---
✓ | - | - | - | - | - | - | 32,9 | 2,877
- | ✓ | - | - | - | - | - | 34,9 | 3,293
- | - | ✓ | - | - | - | - | 35,3 | 3,946
- | - | - | ✓ | - | - | - | 34,2 | 3,19
- | - | - | ✓ | ✓ | - | - | 34,6 | 3,011
- | - | - | ✓ | ✓ | ✓ | - | 34,7 | 2,945
- | - | - | ✓ | ✓ | ✓ | ✓ | 34,1 | 2,823

####  B.4.3 Channels choice

We aimed to determine the optimal number of channels without resorting to
costly training algorithms like NAS, relying instead on iterative
experimentation based on previous architecture choices. Consequently, we
conducted various experiments, including exploring the ideal number of
channels within the FPANet. Table 10 describes all the following explanations.

Backbone channels choice: We desired to optimize channel selection and
discovered that expanding information from P3 to P4 by a factor of 3 rather
than 6, while less potent, required less computation. In this experimentation,
we keeped the strided inverted bottleneck channels expansion ratio of 6 in
between P2 to P3 (from 16 $C_{in}P_{2}$ channels to 96 $C_{mid}P_{3}$) and P4
to P5 (from 64 $C_{in}P_{4}$ to 576 $C_{mid}P_{5}$ channels for LeYOLO-Nano
base channels choice).

Neck channels choice:  Exemplified by our consistent expansion channel choice
of 3 throughout all the models, we tested different expansion ratios inside
the neck with an expansion ratio of 2 and 2.5. With these experimentations, we
wanted to compare the benefits of adding more expansion channels in the
backbone or the neck. Regarding our choices, we compare the neural network’s
performance with more support on the backbone at P5, exemplified by an
expansion ratio of 6, and less pressure on the neck, exemplified by an
expansion ratio of 2 instead of 3 within the whole FPAN neck.

Looking at the ablation proposed in Table 10, we observe that globally,
simplifying the number of channels inside the neck with an expansion ratio of
2 was the better choice as the greater expansion strategy inside the
bottleneck at P5 led to better accuracy and mAP to FLOP ratio.

Table 10: Ablation study on channels choice improvement. $no_{pw}@STEM$ | $no_{pw}@FPAN-P3$ | CC | CCx6 | FPANx2 | FPANx2.5 | mAP | FLOP(G)
---|---|---|---|---|---|---|---
✓ | ✓ | ✓ | - | - | - | 33,5 | 2,75
✓ | ✓ | ✓ | - | ✓ | - | 32,3 | 2,39
✓ | ✓ | ✓ | - | - | ✓ | 32,9 | 2,573
✓ | ✓ | - | - | ✓ | - | 32.6 | 2,467
✓ | ✓ | - | ✓ | ✓ | - | 33,7 | 2,714
✓ | ✓ | ✓ | ✓ | ✓ | - | 34.3 | 2,64

Channel choice:

  1. 1.

CC : Channel choice - we reduce the expansion ratio from P3 to P4 to 3 instead
of 6.

  2. 2.

CCx6 : We augment the expansion ratio inside P5 for all block to 6 instead of
3.

  3. 3.

FPANx2 : All FPAN inverted bottleneck block use a ratio of 2 instead of 3.

  4. 4.

FPANx2.5 : All FPAN inverted bottlenck block use a ratio of 2.5 instead of 3.

Regarding the initial minimal architecture chosen for experimentation, we
began with a 32.9 mAP at 2.877 GFLOP. Following iterative refinements
validated through training experiments, we boosted the accuracy to 34.3 mAP
while reducing computational requirements to 2.64 GFLOP for LeYOLO-Nano. This
signifies an enhancement in accuracy alongside a reduction in computational
complexity during the model refinement process.

####  B.4.4 Architecture scaling choice

This section provides more insight into the model scaling proposed in our
contributions. As discussed in the chapter 2.6.1, we propose four different
scaling methods, compressing the architecture below 10 FLOP(G). Table 11
illustrates the four training scaling possibilities (Nano to Large), and
Table12 shows the eight final proposed scaling for inferences.

Table 11: LeYOLO base training scaling architecture with their respective results Models | Nano | Small | Medium | Large
---|---|---|---|---
Input spatial size | 640 | 640 | 640 | 768
Channels ratio | x1 | x1.33 | x1.33 | x1.33
Layer ratio | x1 | x1 | x1.33 | x1.33
mAP | 34.3 | 38.2 | 39.3 | 41
Table 12: Architecture scaling with different parameter multipliers. Models | Nano | Nano | Small | Small | Small | Medium | Medium | Large
---|---|---|---|---|---|---|---|---
Input spatial size | 320p | 480p | 320p | 480p | 640p | 480p | 640p | 768p
Channels ratio | x1 | x1 | x1.33 | x1.33 | x1.33 | x1.33 | x1.33 | x1.33
Layer ratio | x1 | x1 | x1 | x1 | x1 | x1.33 | x1.33 | x1.33

We can effectively transfer the core architecture from the previously
mentioned ablation study to different input sizes during inference and
validation. For instance, a neural network explicitly trained at 640p might
yield better results when compressed and validated at 320p than a neural
network trained from scratch at 320p. Consequently, we tested various scales
of LeYOLO—Nano, Small, Medium, and Large trained neural network from Table
11—to determine the optimal input and training combinations. The results
presented in Table 13 highlight the best outcomes from this study.

Table 13: Ablation study on best training and validation input size Models | training dim | validation dim | mAP | FLOP(G)
---|---|---|---|---
LeYOLO-Nano | 320 | 320 | 24.1 | 0.66
LeYOLO-Nano | 640 | 320 | 25.2 | 0.66
LeYOLO-Nano | 480 | 480 | 30.9 | 1.47
LeYOLO-Nano | 640 | 480 | 31.3 | 1.47
LeYOLO-Nano(ablation study) | 640 | 640 | 34.3 | 2.65
LeYOLO-Small | 640 | 320 | 29.0 | 1.126
LeYOLO-Small | 640 | 480 | 35.2 | 2.53
LeYOLO-Small | 640 | 640 | 38.2 | 4.51
LeYOLO-Medium | 640 | 320 | 30.0 | 1.45
LeYOLO-Medium | 640 | 480 | 36.4 | 3.27
LeYOLO-Medium | 640 | 640 | 39.3 | 5.8
LeYOLO-Large | 768 | 768 | 41 | 8.4

####  B.4.5 Speed tests

As discussed in the paper, we proposed a highly efficient family of neural
network models, focusing solely on FLOP computation and disregarding execution
speed. Inverted bottlenecks inherently reduce the parallelization potential of
the neural network, causing GPUs to wait for subsequent operations
sequentially. Consequently, while our models may not be the fastest in the
state-of-the-art, they offer various models with varying execution speeds. We
focus on object detectors on embedded devices, so we propose a comparison
using a 4GB Jetson TX2 coupled with the TensorRT software accelerator to
observe the state-of-the-art parallelization capability. We can find details
of execution speed, accuracy, query per second, FLOP, and qps in Table 14 and
Figure 7.

Table 14: Execution speed of reproductible YOLOs and our contribution (sorted by queries per second). Models | QPS | FLOP(G) | mAP(%)
---|---|---|---
LeYOLO-Nano@320 | 99.56 | 0.66 | 25.2
LeYOLO-Small@320 | 75.36 | 1.126 | 29
LeYOLO-Nano@480 | 51.39 | 1.47 | 31.3
LeYOLO-Small@480 | 39.29 | 2.53 | 35.2
YOLOv5n | 38 | 4.5 | 28
YOLOv6n | 37.935 | 11.1 | 35.9
YOLOv8n | 33.650 | 8.7 | 37.3
LeYOLO-Medium@480 | 32.83 | 3.27 | 36.4
YOLOv7-Tiny | 24.8 | 5.8 | 33.3
LeYOLO-Small@640 | 24 | 4.5 | 38.2
LeYOLO-Medium@640 | 19.89 | 5.8 | 39.3
YOLOX-s | 14.6 | 26.8 | 40.5
LeYOLO-Large@768 | 14.2 | 8.4 | 41
Figure 7: YOLO and LeYOLO speed comparison on Jetson TX2 coupled with
TensorRT.

###  B.5 Code

As we could use PyTorch, Tensorflow, or any other API, we are using the
Ultralytics code on the YOLOv8 version to develop our version of LeYOLO. Using
these tools and implementing the code will be simple, centralizing research on
a single tool.

class mn_conv(nn.Module): #Cin, Cout, Kernel, stride, padding, group, dilation
def __init__(self, c1, c2, k=1, s=1, p=None, g=1, d=1): super().__init__()
#autopad -> ultralytics and YOLOv9 auto select padding regarding kernel size
and dilation padding = 0 if k==s else autopad(k,p,d) self.c = nn.Conv2d(c1,
c2, k, s, padding, groups=g) self.bn = nn.BatchNorm2d(c2) self.act = nn.SiLU()

def forward(self, x): return self.act(self.bn(self.c(x)))

class InvertedBottleneck(nn.Module): #Cin, Cout, kernel size, expansion number
(Cin*3 most of the time), stride, first poinwise def __init__(self, c1, c2,
k=3, e=None, stride=1, pw=True): #input_channels, output_channels, repetition,
stride, expension ratio super().__init__() c_mid = e if e != None else c1
self.residual = c1 == c2 and stride == 1

features = [mn_conv(c1, c_mid, act=act)] if pw else [] features.extend([
mn_conv(c_mid, c_mid, k, stride, g=c_mid, act=act), nn.Conv2d(c_mid, c2, 1),
nn.BatchNorm2d(c2), ]) self.layers = nn.Sequential(*features) def
forward(self, x): if self.residual: return x + self.layers(x) else: return
self.layers(x)

class Detect(nn.Module): """LeYOLO Detect head for detection models."""

dynamic = False export = False shape = None anchors = torch.empty(0) # init
strides = torch.empty(0) # init

def __init__(self, nc=80, ch=()): super().__init__() self.nc = nc self.nl =
len(ch) self.reg_max = 16 self.no = nc + self.reg_max * 4 self.stride =
torch.zeros(self.nl)

c2, c3 = max((16, ch[0] // 4, self.reg_max * 4)), max(ch[0], min(self.nc,
100))

self.cv2 = nn.ModuleList( nn.Sequential(Conv(x, c2, 1), Conv(c2, c2, 3, g=c2),
Conv(c2, c2, 3, g=c2), nn.Conv2d(c2, 4 * self.reg_max, 1)) for x in ch )
self.cv3 = nn.ModuleList( nn.Sequential(Conv(x, c3, 1), Conv(c3, c3, 3, g=c3),
Conv(c3, c3, 3, g=c3), nn.Conv2d(c3, self.nc, 1)) for x in ch)

self.dfl = DFL(self.reg_max) if self.reg_max > 1 else nn.Identity()

def forward(self, x):

"""Concatenates and returns predicted bounding boxes and class
probabilities.""" for i in range(self.nl): x[i] =
torch.cat((self.cv2[i](x[i]), self.cv3[i](x[i])), 1) if self.training: #
Training path return x

# Inference path shape = x[0].shape # BCHW x_cat = torch.cat([
xi.view(shape[0], self.no, -1) for xi in x] , 2) if self.dynamic or self.shape
!= shape: self.anchors, self.strides = (x.transpose(0, 1) for x in
make_anchors(x, self.stride, 0.5)) self.shape = shape

dbox = self.decode_bboxes(box)

y = torch.cat((dbox, cls.sigmoid()), 1) return y if self.export else (y, x)

def decode_bboxes(self, bboxes): """Decode bounding boxes.""" return
dist2bbox(self.dfl(bboxes), self.anchors.unsqueeze(0), xywh=True, dim=1) *
self.strides

def dist2bbox(distance, anchor_points, xywh=True, dim=-1): """Transform
distance(ltrb) to box(xywh or xyxy).""" lt, rb = distance.chunk(2, dim) x1y1 =
anchor_points - lt x2y2 = anchor_points + rb if xywh: c_xy = (x1y1 + x2y2) / 2
wh = x2y2 - x1y1 return torch.cat((c_xy, wh), dim) # xywh bbox return
torch.cat((x1y1, x2y2), dim) # xyxy bbox

backbone: # [from, repeats, module, args] \- [-1, 1, mn_conv, [16, 3, 2]] \-
[-1, 1, mn_conv, [16, 1, 1]] \- [-1, 1, InvertedBottleneck, [16, 3, 16, 2,
False]] \- [-1, 1, InvertedBottleneck, [32, 3, 96, 2]] \- [-1, 1,
InvertedBottleneck, [32, 3, 96, 1]] \- [-1, 1, InvertedBottleneck, [64, 5, 96,
2]] \- [-1, 1, InvertedBottleneck, [64, 5, 192, 1]] \- [-1, 1,
InvertedBottleneck, [64, 5, 192, 1]] \- [-1, 1, InvertedBottleneck, [64, 5,
192, 1]] \- [-1, 1, InvertedBottleneck, [64, 5, 192, 1]] \- [-1, 1,
InvertedBottleneck, [96, 5, 576, 2]] \- [-1, 1, InvertedBottleneck, [96, 5,
576, 1]] \- [-1, 1, InvertedBottleneck, [96, 5, 576, 1]] \- [-1, 1,
InvertedBottleneck, [96, 5, 576, 1]] \- [-1, 1, SPPF, [96, 5]]

# LeYOLO FPANet head: \- [-1, 1, nn.Upsample, [None, 2, ’nearest’]] \- [-1, 1,
InvertedBottleneck, [64, 5, 128, 1]] \- [-1, 1, InvertedBottleneck, [64, 5,
128, 1]] \- [-1, 1, InvertedBottleneck, [64, 5, 128, 1]]

\- [-1, 1, nn.Upsample, [None, 2, ’nearest’]] \- [[-1, 4], 1, Concat, [1]] \-
[-1, 1, InvertedBottleneck, [32, 3, 96, 1, False]] \- [-1, 1,
InvertedBottleneck, [32, 3, 64, 1]] \- [-1, 1, InvertedBottleneck, [32, 3, 64,
1]]

\- [-1, 1, mn_conv, [64, 3, 2]] \- [[-1, 19], 1, Concat, [1]] \- [-1, 1,
InvertedBottleneck, [64, 5, 128, 1]] \- [-1, 1, InvertedBottleneck, [64, 5,
128, 1]] \- [-1, 1, InvertedBottleneck, [64, 5, 128, 1]]

\- [-1, 1, mn_conv, [96, 3, 2]] \- [[-1, 14], 1, Concat, [1]] \- [-1, 1,
InvertedBottleneck, [96, 5, 192, 1]] \- [-1, 1, InvertedBottleneck, [96, 5,
192, 1]] \- [-1, 1, InvertedBottleneck, [96, 5, 192, 1]]

#LeYOLO DNiN Head \- [[24, 29, 34], 1, Detect, [nc]] # Detect(P3, P4, P5)

###  B.6 Training specificity

Training on MSCOCO. We train our model on the MSCOCO dataset [37] using the
standard data augmentation [49] with stochastic gradient descent (SGD) and
batch size of 128 on four GPUs. Learning rate is initially set to 0.01 with a
momentum set to 0.9. Weight decay is set to 0.001.

Mosaic data augmentation : throughout the training, we found through multiple
experiments that there is minimal variation in accuracy attributable to Mosaic
data augmentation. This phenomenon primarily arises from small objects with
limited data samples, such as toothbrushes in MSCOCO, where Mosaic
augmentation could potentially have adverse effects. Across our experiments,
we noted a potential variance of 0.4 mAP.

  1. 1.

epochs: 500

  2. 2.

patience: 50

  3. 3.

batch: 128

  4. 4.

imgsz: 640

  5. 5.

gpu count: 4

  6. 6.

workers: 8

  7. 7.

optimizer: SGD

  8. 8.

seed: 0

  9. 9.

close mosaic: 10

  10. 10.

training iou: 0.7

  11. 11.

max detectections: 300

  12. 12.

lr0: 0.01

  13. 13.

lrf: 0.01

  14. 14.

momentum: 0.9

  15. 15.

weight decay: 0.001

  16. 16.

warmup epochs: 3.0

  17. 17.

warmup momentum: 0.8

  18. 18.

warmup bias lr: 0.1

  19. 19.

box: 7.5

  20. 20.

cls: 0.5

  21. 21.

dfl: 1.5

  22. 22.

pose: 12.0

  23. 23.

kobj: 1.0

  24. 24.

label smoothing: 0.0

  25. 25.

nbs: 64

  26. 26.

hsv h: 0.015

  27. 27.

hsv s: 0.7

  28. 28.

hsv v: 0.4

  29. 29.

degrees: 0.0

  30. 30.

translate: 0.1

  31. 31.

scale: 0.5

  32. 32.

shear: 0.0

  33. 33.

perspective: 0.0

  34. 34.

flipud: 0.0

  35. 35.

fliplr: 0.5

  36. 36.

mosaic: 1.0

  37. 37.

mixup: 0.0

  38. 38.

copy paste: 0.0

  39. 39.

erasing: 0.4

  40. 40.

crop fraction: 1.0

## NeurIPS Paper Checklist

  1. 1.

Claims

  2. Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

  3. Answer: [Yes]

  4. Justification: The abstract and introduction clearly state the purpose of neural networks with low computational resources for embedded devices and support its interest and the interest of YOLO for the industry. The introduction includes evidence of a contribution by the results (accuracy, precision, state-of-the-art comparison) and the contribution limitations (fully discussed in the discussion section.)

  5. Guidelines:

     * •

The answer NA means that the abstract and introduction do not include the
claims made in the paper.

     * •

The abstract and/or introduction should clearly state the claims made,
including the contributions made in the paper and important assumptions and
limitations. A No or NA answer to this question will not be perceived well by
the reviewers.

     * •

The claims made should match theoretical and experimental results, and reflect
how much the results can be expected to generalize to other settings.

     * •

It is fine to include aspirational goals as motivation as long as it is clear
that these goals are not attained by the paper.

  6. 2.

Limitations

  7. Question: Does the paper discuss the limitations of the work performed by the authors?

  8. Answer: [Yes]

  9. Justification: The paper demonstrates the robustness of certain results through an ablation study on specific datasets. In the discussion chapter, we openly acknowledge our contributions’ limitations, addressing the flaws and what is missing. We also discuss the computational efficiency of the proposed algorithms, advancing the boundaries of the FLOP to mAP ratio. However, we admit that our contributions may require speed improvements, as they are not the fastest in the state-of-the-art.

  10. Guidelines:

     * •

The answer NA means that the paper has no limitation while the answer No means
that the paper has limitations, but those are not discussed in the paper.

     * •

The authors are encouraged to create a separate "Limitations" section in their
paper.

     * •

The paper should point out any strong assumptions and how robust the results
are to violations of these assumptions (e.g., independence assumptions,
noiseless settings, model well-specification, asymptotic approximations only
holding locally). The authors should reflect on how these assumptions might be
violated in practice and what the implications would be.

     * •

The authors should reflect on the scope of the claims made, e.g., if the
approach was only tested on a few datasets or with a few runs. In general,
empirical results often depend on implicit assumptions, which should be
articulated.

     * •

The authors should reflect on the factors that influence the performance of
the approach. For example, a facial recognition algorithm may perform poorly
when image resolution is low or images are taken in low lighting. Or a speech-
to-text system might not be used reliably to provide closed captions for
online lectures because it fails to handle technical jargon.

     * •

The authors should discuss the computational efficiency of the proposed
algorithms and how they scale with dataset size.

     * •

If applicable, the authors should discuss possible limitations of their
approach to address problems of privacy and fairness.

     * •

While the authors might fear that complete honesty about limitations might be
used by reviewers as grounds for rejection, a worse outcome might be that
reviewers discover limitations that aren’t acknowledged in the paper. The
authors should use their best judgment and recognize that individual actions
in favor of transparency play an important role in developing norms that
preserve the integrity of the community. Reviewers will be specifically
instructed to not penalize honesty concerning limitations.

  11. 3.

Theory Assumptions and Proofs

  12. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

  13. Answer: [N/A]

  14. Justification: In Chapter 2.3.1, we provide theoretical ideas on the optimization and compression of neural networks, including the choice of layers and number of channels. In the context of deep learning, it is difficult today to mathematically prove the choice of the number of layers, channels, and parameters with solid and unbreakable proof. Nevertheless, we strive to use experimental proof with as many samples as possible and ablation study.

  15. Guidelines:

     * •

The answer NA means that the paper does not include theoretical results.

     * •

All the theorems, formulas, and proofs in the paper should be numbered and
cross-referenced.

     * •

All assumptions should be clearly stated or referenced in the statement of any
theorems.

     * •

The proofs can either appear in the main paper or the supplemental material,
but if they appear in the supplemental material, the authors are encouraged to
provide a short proof sketch to provide intuition.

     * •

Inversely, any informal proof provided in the core of the paper should be
complemented by formal proofs provided in appendix or supplemental material.

     * •

Theorems and Lemmas that the proof relies upon should be properly referenced.

  16. 4.

Experimental Result Reproducibility

  17. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

  18. Answer: [Yes]

  19. Justification: Our contribution uses experiments and is reproducible in many ways. We provide a code to quickly launch our model. Also, our model’s flexible architecture easily integrates with Ultralytics (YOLOv8) or YOLOv9 repository for training and inference purposes.

  20. Guidelines:

     * •

The answer NA means that the paper does not include experiments.

     * •

If the paper includes experiments, a No answer to this question will not be
perceived well by the reviewers: Making the paper reproducible is important,
regardless of whether the code and data are provided or not.

     * •

If the contribution is a dataset and/or model, the authors should describe the
steps taken to make their results reproducible or verifiable.

     * •

Depending on the contribution, reproducibility can be accomplished in various
ways. For example, if the contribution is a novel architecture, describing the
architecture fully might suffice, or if the contribution is a specific model
and empirical evaluation, it may be necessary to either make it possible for
others to replicate the model with the same dataset, or provide access to the
model. In general. releasing code and data is often one good way to accomplish
this, but reproducibility can also be provided via detailed instructions for
how to replicate the results, access to a hosted model (e.g., in the case of a
large language model), releasing of a model checkpoint, or other means that
are appropriate to the research performed.

     * •

While NeurIPS does not require releasing code, the conference does require all
submissions to provide some reasonable avenue for reproducibility, which may
depend on the nature of the contribution. For example

       1. (a)

If the contribution is primarily a new algorithm, the paper should make it
clear how to reproduce that algorithm.

       2. (b)

If the contribution is primarily a new model architecture, the paper should
describe the architecture clearly and fully.

       3. (c)

If the contribution is a new model (e.g., a large language model), then there
should either be a way to access this model for reproducing the results or a
way to reproduce the model (e.g., with an open-source dataset or instructions
for how to construct the dataset).

       4. (d)

We recognize that reproducibility may be tricky in some cases, in which case
authors are welcome to describe the particular way they provide for
reproducibility. In the case of closed-source models, it may be that access to
the model is limited in some way (e.g., to registered users), but it should be
possible for other researchers to have some path to reproducing or verifying
the results.

  21. 5.

Open access to data and code

  22. Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

  23. Answer: [Yes]

  24. Justification: We make our code available. As for the data, we are only working on MSCOCO for this study.

  25. Guidelines:

     * •

The answer NA means that paper does not include experiments requiring code.

     * •

Please see the NeurIPS code and data submission guidelines
(https://nips.cc/public/guides/CodeSubmissionPolicy) for more details.

     * •

While we encourage the release of code and data, we understand that this might
not be possible, so “No” is an acceptable answer. Papers cannot be rejected
simply for not including code, unless this is central to the contribution
(e.g., for a new open-source benchmark).

     * •

The instructions should contain the exact command and environment needed to
run to reproduce the results. See the NeurIPS code and data submission
guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more
details.

     * •

The authors should provide instructions on data access and preparation,
including how to access the raw data, preprocessed data, intermediate data,
and generated data, etc.

     * •

The authors should provide scripts to reproduce all experimental results for
the new proposed method and baselines. If only a subset of experiments are
reproducible, they should state which ones are omitted from the script and
why.

     * •

At submission time, to preserve anonymity, the authors should release
anonymized versions (if applicable).

     * •

Providing as much information as possible in supplemental material (appended
to the paper) is recommended, but including URLs to data and code is
permitted.

  26. 6.

Experimental Setting/Details

  27. Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

  28. Answer: [Yes]

  29. Justification: We show all the settings used to train the model. We have mainly based our study on "simple" learning techniques to focus on neural network architectures, although we know the wide range of training possibilities. As a result, readers don’t have to worry about complex training methods and hyperparameters, as we use the basic YOLO training parameters on versions 5, 8, and 9.

  30. Guidelines:

     * •

The answer NA means that the paper does not include experiments.

     * •

The experimental setting should be presented in the core of the paper to a
level of detail that is necessary to appreciate the results and make sense of
them.

     * •

The full details can be provided either with the code, in appendix, or as
supplemental material.

  31. 7.

Experiment Statistical Significance

  32. Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

  33. Answer: [Yes]

  34. Justification:  During our experiments, we based the importance of our results on a rigorous comparison with the state of the art. To achieve this comparison, we have included diagrams, plots, and other information relevant to understanding the experiments, both in the paper and in the appendix.

  35. Guidelines:

     * •

The answer NA means that the paper does not include experiments.

     * •

The authors should answer "Yes" if the results are accompanied by error bars,
confidence intervals, or statistical significance tests, at least for the
experiments that support the main claims of the paper.

     * •

The factors of variability that the error bars are capturing should be clearly
stated (for example, train/test split, initialization, random drawing of some
parameter, or overall run with given experimental conditions).

     * •

The method for calculating the error bars should be explained (closed form
formula, call to a library function, bootstrap, etc.)

     * •

The assumptions made should be given (e.g., Normally distributed errors).

     * •

It should be clear whether the error bar is the standard deviation or the
standard error of the mean.

     * •

It is OK to report 1-sigma error bars, but one should state it. The authors
should preferably report a 2-sigma error bar than state that they have a 96%
CI, if the hypothesis of Normality of errors is not verified.

     * •

For asymmetric distributions, the authors should be careful not to show in
tables or figures symmetric error bars that would yield results that are out
of range (e.g. negative error rates).

     * •

If error bars are reported in tables or plots, The authors should explain in
the text how they were calculated and reference the corresponding figures or
tables in the text.

  36. 8.

Experiments Compute Resources

  37. Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

  38. Answer: [Yes]

  39. Justification: Although our contribution uses a more global comparison of the computational cost of a neural network (MAC and FLOP), we are concerned with the speed of execution of such a model around YOLO architectures. To do this, we are running our contribution on one embedded device: Jetson TX2 4GB. For the speed benchmark, we have indicated the hardware used.

  40. Guidelines:

     * •

The answer NA means that the paper does not include experiments.

     * •

The paper should indicate the type of compute workers CPU or GPU, internal
cluster, or cloud provider, including relevant memory and storage.

     * •

The paper should provide the amount of compute required for each of the
individual experimental runs as well as estimate the total compute.

     * •

The paper should disclose whether the full research project required more
compute than the experiments reported in the paper (e.g., preliminary or
failed experiments that didn’t make it into the paper).

  41. 9.

Code Of Ethics

  42. Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?

  43. Answer: [Yes]

  44. Justification: The paper does not violate the requirements of the Code of Ethics. Also, we preserve anonymity in the paper and the provided codes.

  45. Guidelines:

     * •

The answer NA means that the authors have not reviewed the NeurIPS Code of
Ethics.

     * •

If the authors answer No, they should explain the special circumstances that
require a deviation from the Code of Ethics.

     * •

The authors should make sure to preserve anonymity (e.g., if there is a
special consideration due to laws or regulations in their jurisdiction).

  46. 10.

Broader Impacts

  47. Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

  48. Answer: [Yes]

  49. Justification: YOLO has a significant societal impact, especially in the edge, embedded, and industrial sectors. This is partly due to previous work enabling YOLO to be used rapidly by most scientists (even outside computer science and AI). Consequently, proposing an alternative to YOLO, based mainly on cost optimization, enables readers to get a high-performance alternative (mAP) for less computing resources than the new YOLO (v7,v8,v9).

  50. Guidelines:

     * •

The answer NA means that there is no societal impact of the work performed.

     * •

If the authors answer NA or No, they should explain why their work has no
societal impact or why the paper does not address societal impact.

     * •

Examples of negative societal impacts include potential malicious or
unintended uses (e.g., disinformation, generating fake profiles,
surveillance), fairness considerations (e.g., deployment of technologies that
could make decisions that unfairly impact specific groups), privacy
considerations, and security considerations.

     * •

The conference expects that many papers will be foundational research and not
tied to particular applications, let alone deployments. However, if there is a
direct path to any negative applications, the authors should point it out. For
example, it is legitimate to point out that an improvement in the quality of
generative models could be used to generate deepfakes for disinformation. On
the other hand, it is not needed to point out that a generic algorithm for
optimizing neural networks could enable people to train models that generate
Deepfakes faster.

     * •

The authors should consider possible harms that could arise when the
technology is being used as intended and functioning correctly, harms that
could arise when the technology is being used as intended but gives incorrect
results, and harms following from (intentional or unintentional) misuse of the
technology.

     * •

If there are negative societal impacts, the authors could also discuss
possible mitigation strategies (e.g., gated release of models, providing
defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms
to monitor how a system learns from feedback over time, improving the
efficiency and accessibility of ML).

  51. 11.

Safeguards

  52. Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

  53. Answer: [N/A]

  54. Justification: Our work contributes to object detection within a very carefully designed dataset: MSCOCO. The paper does not pose such risks.

  55. Guidelines:

     * •

The answer NA means that the paper poses no such risks.

     * •

Released models that have a high risk for misuse or dual-use should be
released with necessary safeguards to allow for controlled use of the model,
for example by requiring that users adhere to usage guidelines or restrictions
to access the model or implementing safety filters.

     * •

Datasets that have been scraped from the Internet could pose safety risks. The
authors should describe how they avoided releasing unsafe images.

     * •

We recognize that providing effective safeguards is challenging, and many
papers do not require this, but we encourage authors to take this into account
and make a best faith effort.

  56. 12.

Licenses for existing assets

  57. Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

  58. Answer: [Yes]

  59. Justification: We have appropriately cited all assets (datasets and code) used in our research. Our work builds upon the open-source AGPL-3.0 licensed YOLOv8 code from Ultralytics. Throughout the paper, we emphasize that we are using code from Ultralytics, highlighting that it is easily reproducible due to the availability of this library.

  60. Guidelines:

     * •

The answer NA means that the paper does not use existing assets.

     * •

The authors should cite the original paper that produced the code package or
dataset.

     * •

The authors should state which version of the asset is used and, if possible,
include a URL.

     * •

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

     * •

For scraped data from a particular source (e.g., website), the copyright and
terms of service of that source should be provided.

     * •

If assets are released, the license, copyright information, and terms of use
in the package should be provided. For popular datasets,
paperswithcode.com/datasets has curated licenses for some datasets. Their
licensing guide can help determine the license of a dataset.

     * •

For existing datasets that are re-packaged, both the original license and the
license of the derived asset (if it has changed) should be provided.

     * •

If this information is not available online, the authors are encouraged to
reach out to the asset’s creators.

  61. 13.

New Assets

  62. Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

  63. Answer: [Yes]

  64. Justification: We have provided the code necessary to build our neural network using the YOLOv8 or YOLOv9 API, along with a comprehensive overview of the architecture schema.

  65. Guidelines:

     * •

The answer NA means that the paper does not release new assets.

     * •

Researchers should communicate the details of the dataset/code/model as part
of their submissions via structured templates. This includes details about
training, license, limitations, etc.

     * •

The paper should discuss whether and how consent was obtained from people
whose asset is used.

     * •

At submission time, remember to anonymize your assets (if applicable). You can
either create an anonymized URL or include an anonymized zip file.

  66. 14.

Crowdsourcing and Research with Human Subjects

  67. Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

  68. Answer: [N/A]

  69. Justification: The paper does not involve crowdsourcing or research with human subjects.

  70. Guidelines:

     * •

The answer NA means that the paper does not involve crowdsourcing nor research
with human subjects.

     * •

Including this information in the supplemental material is fine, but if the
main contribution of the paper involves human subjects, then as much detail as
possible should be included in the main paper.

     * •

According to the NeurIPS Code of Ethics, workers involved in data collection,
curation, or other labor should be paid at least the minimum wage in the
country of the data collector.

  71. 15.

Institutional Review Board (IRB) Approvals or Equivalent for Research with
Human Subjects

  72. Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

  73. Answer: [N/A]

  74. Justification: The paper does not involve crowdsourcing or research with human subjects.

  75. Guidelines:

     * •

The answer NA means that the paper does not involve crowdsourcing nor research
with human subjects.

     * •

Depending on the country in which research is conducted, IRB approval (or
equivalent) may be required for any human subjects research. If you obtained
IRB approval, you should clearly state this in the paper.

     * •

We recognize that the procedures for this may vary significantly between
institutions and locations, and we expect authors to adhere to the NeurIPS
Code of Ethics and the guidelines for their institution.

     * •

For initial submissions, do not include any information that would break
anonymity (if applicable), such as the institution conducting the review.