<p>Depends on the ensembling method; it's an active area of research I suggest you look into, but I'll provide some examples below:</p>

<ul>
<li><strong>Dropout</strong>: trains parts of the model at any given iterations, thus effectively training a multi-NN ensemble</li>
<li><a href="https://www.wikiwand.com/en/Ensemble_averaging_(machine_learning)" rel="nofollow noreferrer"><strong>Weights averaging</strong></a>: train X models on X different splits of data to learn different features, average the early-stopped weights (requires advanted treatment)</li>
<li><a href="https://arxiv.org/abs/1907.08610" rel="nofollow noreferrer"><strong>Lookahead optimizer</strong></a>: automates the above by performing the averaging during training</li>
<li><strong>Parallel weak learners</strong>: run X models, but each model taking 1/X the time to process - which can be achieved by e.g. inserting a <code>strides=X</code> convolutional layer at input; best starting bet is at X=2, so you'll average two models' predictions at output, each prediction made in parallel (which can run <em>faster</em> than original single model)</li>
</ul>

<p>If you have a multi-core CPU, however, multi-model ensembling shouldn't pose much of a problem, as per last bullet, you can run inference concurrently, so inference time shouldn't increase much</p>

<hr>

<p><strong>More on parallelism</strong>: if a single model is large enough, CPU parallelism will no longer help - you'll also need to ensure multiple models can fit in memory (RAM). The alternative then is again a form of downsampling to cut computation</p>
Can we create an ensemble of deep learning models without increasing the classification time?
<p>I want to improve my ResNet model by creating an ensemble of X number of this model, taking the X best one I have trained. For what I've seen, a technique like bagging will take X time longer to classify an image, which is really not an option in my case. </p>

<p>Is there a way to create an ensemble without increasing the required classifying time? Note that I don't care about increasing the training time, because it only needs to be done one time, compared to the classification which could be made a very large number of time. </p>