<p>I will assume that you have already double, triple and quadruple checked that the data going in is matching what you expect.</p>

<hr>

<p>The question is quite open-ended, and even a topic for research. But there are some things that can help.</p>

<p>In terms of better training, there's two normal ways in which people train neural networks with an unbalanced dataset.</p>

<ul>
<li>Oversample the examples with lower frequency, such that the proportion of examples for each class that the network sees is equal. e.g. in every batch, enforce that 1/4 of the examples are from class 1, 1/4 from class 2, etc.</li>
<li>Weight the error for misclassifying each class by it's proportion. e.g. incorrectly classifying an example of class 1 is worth 100/43, while incorrectly classifying an example of class 4 is worth 100/7</li>
</ul>

<p>That being said, if your learning rate is good, neural networks will often eventually (after many hours of just sitting there) jump out of only predicting for one class, but they still rarely end well with a badly skewed dataset.</p>

<hr>

<p>If you want to know whether or not there <em>are</em> patterns in your data which can be determined, there is a simple way to do that. </p>

<p>Create a new dataset by randomly select elements from all of your classes such that you have an even number of all of them (i.e. if there's 700 examples of class 4, then construct a dataset by randomly selecting 700 examples from every class)</p>

<p>Then you can use all of your techniques on this new dataset.</p>

<p>Although, <a href="https://arxiv.org/abs/1611.03530" rel="nofollow noreferrer">this paper</a> suggests that even with random labels, it should be able to find some pattern that it understands.</p>
Neural Network gets stuck
<p>I am experimenting with classification using neural networks (I am using tensorflow).
And unfortunately the training of my neural network gets stuck at 42% accuracy.
I have 4 classes, into which I try to classify the data.
And unfortunately, my data set is not well balanced, meaning that:</p>

<ol>
<li>43% of the data belongs to class 1 (and yes, my network gets stuck  predicting only this)</li>
<li>37% to class 2</li>
<li>13% to class 3</li>
<li>7% to class 4</li>
</ol>

<p>The optimizer I am using is AdamOptimizer and the cost function is tf.nn.softmax_cross_entropy_with_logits.</p>

<p>I was wondering if the reason for my training getting stuck at 42% is really the fact that my data set is not well balanced, or because the nature of the data is really random, and there are really no patterns to be found.</p>

<p>Currently my NN consists of:</p>

<ol>
<li>input layer </li>
<li>2 convolution layers </li>
<li>7 fully connected layers</li>
<li>output layer</li>
</ol>

<p>I tried changing this structure of the network, but the result is always the same.
I also tried Support Vector Classification, and the result is pretty much the same, with small variations.</p>

<p>Did somebody else encounter similar problems?
Could anybody please provide me some hints how to get out of this issue?</p>

<p>Thanks,
Gerald</p>