<p>You may learn an auto-encoder on the negative examples you have (if their number is kind of large) and then generate examples using an inference technique such as <a href="http://arxiv.org/abs/1312.6114" rel="nofollow">variational Bayes</a> or <a href="http://jmlr.csail.mit.edu/papers/volume15/alain14a/alain14a.pdf" rel="nofollow">Markov Chain Monte Carlo</a>. This way you can increase the number of samples for the negative examples and kind of move towards a more balanced data set.</p>
Tensorflow classification with extremely unbalanced dataset
<p>I am using TensorFlow LinearClassifier and also DNN to classify two - classes dataset.</p>

<p>However, the problem is the dataset contains 96% of Positive output, and 4% of negative output, and my program always return the prediction as Positive. Of course, in this case I will achieved the accuracy of 96%, but it does not make sense at all.</p>

<p>What is the good way to deal with this kind of situation?</p>