<blockquote>
<p>Change is number of classes doesn't have significant impact on
inference time.</p>
</blockquote>
<p>For example in case of <a href="https://github.com/AlexeyAB/darknet/blob/master/cfg/yolov4.cfg" rel="nofollow noreferrer">Yolov4</a>, which has got 3 <code>Yolo</code> layers, change in classes leads to change in <strong><code>filter size</code></strong> for <code>conv</code> layers preceding <code>Yolo</code> layers and some computation reduction within <code>Yolo</code> layers that's all. This is very minute compared to overall inference time as <code>conv</code> layers preceding <code>Yolo</code> layers are bottom layers with very small width and hight and also time spent on logic that depends upon number of classes within <a href="https://github.com/AlexeyAB/darknet/blob/master/src/region_layer.c" rel="nofollow noreferrer">Yolo</a> layer is very less.</p>
<p><a href="https://i.stack.imgur.com/Y62zU.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Y62zU.png" alt="Snapshot of Yolo layer" /></a></p>
<p>Here:</p>
<pre><code>filters=(classes + 5)x3
</code></pre>
<p>Note that tinier version of yolov4 i.e <a href="https://github.com/AlexeyAB/darknet/blob/master/cfg/yolov4-tiny.cfg" rel="nofollow noreferrer">tiny-yolov4</a> have got two <code>Yolo</code> layers only, instead of 3.</p>
<p><strong>If your intent is to reduce inference time, especially on raspberry pi or a jetson nano, without losing on accuracy/mAP, do following things:</strong></p>
<ul>
<li><p><strong>Quantisation</strong>: Run inference with <code>INT8</code> instead of <code>FP32</code>. You can use this <a href="https://github.com/AlexeyAB/yolo2_light" rel="nofollow noreferrer">repo</a> for this purpose. You can do this for both Jetson nano and raspberry pi.</p>
</li>
<li><p>Use inference library such as <a href="https://github.com/ceccocats/tkDNN" rel="nofollow noreferrer"><strong>tkDNN</strong></a>, which is a Deep Neural Network library built with <code>cuDNN</code> and <code>tensorRT</code> primitives, specifically thought to work on <strong>NVIDIA Jetson</strong> Boards. You can use this for Jetson nano. Note that with <a href="https://developer.nvidia.com/tensorrt" rel="nofollow noreferrer"><strong>TensorRT</strong></a>, you can use <code>INT8</code> and <code>FP16</code> instead of <code>FP32</code> to reduce detection time.</p>
</li>
</ul>
<p><strong>Following techniques can be used to reduce inference time, but they come at the cost of significant drop in accuracy/mAP:</strong></p>
<ul>
<li>You can train the models with tinier versions rather than full Yolo versions.</li>
<li>Model Pruning - If you could rank the neurons in the network according to how much they contribute, you could then remove the low ranking neurons from the network, resulting in a smaller and faster network. Pruned yolov3 research <a href="https://arxiv.org/abs/1907.11093v1" rel="nofollow noreferrer">paper</a> and it's <a href="https://github.com/PengyiZhang/SlimYOLOv3" rel="nofollow noreferrer">implementation</a>. <a href="https://github.com/Lam1360/YOLOv3-model-pruning" rel="nofollow noreferrer">This</a> is another pruned Yolov3 implementation.</li>
</ul>
Is it possible to significantly reduce the inference time of images by reducing the number of object classes?
<p>I am using YOLOv4 to train my custom detector. Source: <a href="https://github.com/AlexeyAB/darknet" rel="nofollow noreferrer">https://github.com/AlexeyAB/darknet</a></p>
<p>One of the issues while training is the computing power of GPU and available video RAM. What is the relationship between number of object classes and the time it takes to train the model? Also, is it possible to significantly reduce the inference time of images by reducing the number of object classes? The goal is to run inference on a Raspberry Pi or a Jetson Nano.</p>
<p>Any help is much appreciated. Thanks.</p>