<p>Fully connected layers can indeed be very heavy. Please look at section "3.1 Truncated SVD for faster detection" at <a href="https://arxiv.org/abs/1504.08083" rel="nofollow noreferrer"><em>Girshick, R</em> <strong>Fast-RCNN</strong> ICCV 2015</a> describing how to use <a href="https://en.wikipedia.org/wiki/Singular_value_decomposition" rel="nofollow noreferrer">SVD</a> trick to <em>significalntly</em> reduce the burden of fully connected layers. Hence, you can replace your three fully connected layers with 6 very thin layers.</p>

<p>Steps to go from model <code>A</code> to <code>B</code>:  </p>

<ol>
<li><p>Create <code>B.prototxt</code> that has the 5 convolution layers <strong>with the same <code>"name"</code>s</strong> as <code>A</code>.  </p></li>
<li><p>Give the single fully connected layer in <code>B</code> a new <code>"name"</code> that does not exist in <code>A</code>.</p></li>
<li><p>in python  </p>

<pre><code>import caffe
B = caffe.Net('/path/to/B.prototxt', '/path/to/weights_A.caffemodel', caffe.TEST)
B.save('/path/to/weights_B.caffemodel')
</code></pre></li>
<li><p>Now you have weights for <code>B</code> that are the same as the weights of <code>A</code> for all convolutional layers and <strong>random</strong> for the new single fully connected layer.</p></li>
<li><p>fine tune model <code>B</code> starting from <code>'/path/to/weights_B.caffemodel'</code> to learn the weights for the new single fully connected layer.</p></li>
</ol>
Layer drop and update caffe model
<p>I need to update a caffe model from an existing caffe model where I will drop last two layers. It is needed to reduce caffe model size so that it would be easier and lesser size to deploy. Say my existing caffe model is <strong>A1.caffemodel</strong> which has <strong><em>5 convolution layers</em></strong> and <strong><em>3 fully connected layers</em></strong>. I want to generate a new model from it named <strong>B1.caffemodel</strong> which will have <strong><em>5 convolution layers</em></strong> and <strong><em>1 fully connected layer</em></strong> (last 2 fc layers discarded).</p>

<p>I appreciate your all valuable suggestions and helpful code snippet. </p>