<p>If you look inside the "black box" of models such as Yolo and Mask-RCNN, you will realize that they already contain "multiple small networks", to a certain extent, regarding object detection. </p>

<p>Actually, Mask-RCNN is roughly a Faster-RCNN with an additional branch for segmentation. However, regarding detection, there is "somewhere" a classification layer that give a score for each class object (and a regression layer to estimate the box). All the object classes are estimated from a common representation (all the rest of the network) and only the last layer is specialized to each class. The point is nevertheless that there are advantages to compute the common representation jointly to all object classes, in particular because a positive sample for class <code>i</code> is usually also a negative sample for class <code>j</code>.</p>

<p>The idea is quite different for YOLO (v1) but "somewhere" at the end of the network, there is a stack of neurons layers. There is a layer for each object class and it computes the probability of presence of the corresponding object in a region of the image. Once again, the layers are computed from a "common representation" thus in that sense they are quite independent "classifiers". But once again, these "classifiers" benefit from the representation that is computed in jointly for all class objects.</p>

<p>To be honest, these explanation are quite approximate, in order to try to be clear. If you really want to understand, the best is to read <a href="https://pjreddie.com/publications/" rel="nofollow noreferrer">the publication(s) of Yolo</a> or <a href="https://arxiv.org/abs/1703.06870" rel="nofollow noreferrer">that of mask R-CNN</a>. However, it is quite technical and requires to understand quite well deep learning basics. There are also some good tutorials on the web.</p>

<p>This being said, you can modify the architecture of Yolo and Mask R-CNN to put more complex "small neural networks" in replacement of the existing layers. It may improve performance since you will have more neurons, but will be also more complex to train. As said in comments by @jakub, you can also train multiple specific network and add a layer to choose between all, but it would be a "new" architecture and I doubt that you will obtain a better compromise between performances and computational efficiency than Yolo or Mask R-CNN</p>
Can we use multiple models for object detection?
<p>So I have been working on object detection since a lot of time, I have seen models like YOLO and Mask-RCNN use a single deep model to classify objects. Is it possible to make multiple small networks comparatively to identify each and every object separately to increase accuracy and what will be the effect on speed. I'm confused little bit.</p>