<p>Another -- and drastically better -- option is to not use VGG16.  If you look at <a href="https://arxiv.org/pdf/1707.07012.pdf" rel="nofollow noreferrer">Figure 5 in this paper</a>, you'll note that VGG16 does very badly in terms of accuracy vs. FLOPs (floating point operations per second).  If you need speed, Mobilenet or a reduced-size ResNet will do much better.  Even inception-v2 will outperform VGG in accuracy with much lower computational cost.</p>

<p>This will drastically reduce your training time <em>and</em> memory use.</p>
Fine-tuning VGG last layer is very slow
<p>I am fine-tuning a VGG16 network on 32 cpu machine using tensorflow. I used cross entropy loss with sparse. I have to classify the cloths images into 50 classes. After 2 weeks of training this is how the loss is going down, which I feel is very slow convergence. My batch size is 50. Is it normal or what do you think is going wrong here? Accuracy is also really bad. And now it crashed with bad memory allocation error.
<code>terminate called after throwing an instance of 'std::bad_alloc' what():  std::bad_allo</code></p>

<p>My last line in log file looks like this - </p>

<p><code>2016-12-13 08:56:57.162186: step 31525, loss = 232179.64 (1463843.280 sec/batch)</code></p>

<p><a href="https://i.stack.imgur.com/I4A52.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/I4A52.png" alt="total loss"></a></p>

<p>I also tried Tesla K80 GPU and after 20 hrs of training this is how the loss looks like. All parameters are same. Worrying part is - using GPU didn't increase the iteration rate which means each step is taking same time either in  32 cpu with 50 threds or in tesla K80. </p>

<p>I definitely need some practical advice here.</p>

<p><a href="https://i.stack.imgur.com/QKA13.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/QKA13.png" alt="total loss GPU"></a>
<a href="https://i.stack.imgur.com/pY5MF.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/pY5MF.png" alt="enter image description here"></a></p>