<p>A few suggestions:</p>

<ol>
<li>Change initialization from <code>gauss</code> to <code>xavier</code>.</li>
<li>Work with <a href="http://arxiv.org/abs/1502.01852" rel="nofollow"><code>"PReLU"</code></a> acitvations, instead of <code>"ReLU"</code>. once your net converges you can finetune to remove them.  </li>
<li>Try reducing <code>base_lr</code> by an order of magnitude (or even two orders).</li>
</ol>
Test accuracy cannot improve when learning ZFNet on ILSVRC12
<p>I've implemented a home-brewed ZFNet (<a href="https://gist.github.com/stoneyang/d9f636ab78ee325db51b2822e71f4011" rel="nofollow">prototxt</a>) for my research. After 20k iterations with the definition, the test accuracy stays at ~0.001 (i.e., 1/1000), the test loss at ~6.9, and training loss at ~6.9, which seems that the net keeps playing guessing games among the 1k classes. I've thoroughly checked the whole definition and tried to change some of the hyper-parameters to start a new training, but of no avail, same results' shown on the screen....</p>

<p>Could anyone show me some light? Thanks in advance! </p>

<hr>

<p>The hyper-parameters in the prototxt are derived from the paper [1]. All the inputs and outputs of the layers seems correct as Fig. 3 in the paper suggests. </p>

<p>The tweaks are: </p>

<ul>
<li><p><code>crop</code>-s of the input for both training and testing are set to <code>225</code> instead of <code>224</code> as discussed in #33;</p></li>
<li><p>one-pixel zero paddings for <code>conv3</code>, <code>conv4</code>, and <code>conv5</code> to make the sizes of the blobs consistent [1]; </p></li>
<li><p>filler types for all learnable layers changed from <code>constant</code> in [1] to <code>gaussian</code> with <code>std: 0.01</code>; </p></li>
<li><p><code>weight_decay</code>: changing from <code>0.0005</code> to <code>0.00025</code> as suggested by @sergeyk in PR #33;</p></li>
</ul>

<p>[1] Zeiler, M. and Fergus, R. Visualizing and Understanding Convolutional Networks, ECCV 2014. </p>

<p>and for the poor part..., I pasted it <a href="http://pastebin.com/hCXBAfwm" rel="nofollow">here</a></p>