title: Idea behind how many fully-connected layers should be use in a general CNN network
body: <p>I notice that some famous CNN structure in ILSVRC, such as AlexNet, VGG, ZF net, etc. They all use two fully-connected layer, followed by the output layer.
  So why two? Is there any in intrinsic idea behind this?</p>
  
  <p>I try to understand it in this way: before the fully-connected layer, we have bunch of convolutional layers, which might contains various high-level features. And the fully-connected layer is something like a feature list abstracted from convoluted layers. But in this sense, one FC layer should be enough. Why two? And why not three or four or more? I guess a constrains behind this might be computing cost. But is it necessary that more FC layer always provide better result? And what might be the reason for choosing two?</p>