title: What does "sparse" mean in the context of neural nets?
body: <p>I've seen "sparse" and "sparsity" used in a way that suggests it's something that improves a model's accuracy. For example:</p>
  
  <blockquote>
    <p>I think the unsupervised phase might be not so important if some
    sparse connections or neurons are used, such as rectifier units or
    convolutional connection, and big training data is available.</p>
  </blockquote>
  
  <p>From <a href="https://www.quora.com/When-does-unsupervised-pre-training-improve-classification-accuracy-for-a-deep-neural-network-When-does-it-not" rel="noreferrer">https://www.quora.com/When-does-unsupervised-pre-training-improve-classification-accuracy-for-a-deep-neural-network-When-does-it-not</a></p>
  
  <p>What does "sparse" mean in this context?</p>