title: Why does Weka RandomForest prediction differ from validation?
body: <p>I just started to use Weka several weeks ago for a land cover classification of remotely sensed data. I'm no Data Mining expert, but until now, everything worked properly. By the way, I'm using Weka with cmd (.bat-files).</p>
  
  <p>Now I'm a little bit helpless, also because it seems that my problem does not exist at all throughout the web. My problem is, that one class is missing in the prediction data, but was previously correctly classified during validation process (at least for a few instances). Is that possible? And why? More specific:</p>
  
  <p>I've done RandomForest prediction by using <em>weka.filters.supervised.attribute.AddClassification</em>, as it is shown on <a href="http://weka.wikispaces.com/Making+predictions" rel="nofollow">http://weka.wikispaces.com/Making+predictions</a>. Previously, I trained the classifier with a training set, and validated with a separate test set (63 features, training: ~300.000 instances, validation: ~ 100.000 instances). I have 13 different classes. Not every class shows satisfying accuracies, but TP Rate and Precision are NEVER Zero. The .model-file that came out of this process was loaded in the above mentioned prediction. My "prediction set" contains a huge amount of data (Rasterstack with ~ 16 Mio. Pixels, therefore splitted during arff-Conversion). As is the training set, the test set is part of this prediction set. But my resulting classification contains every class except one... am I completely wrong or should the result contain EVERY class - at least in the validation areas? Or asked differently: Should'nt the results of <em>weka.classifiers.trees.RandomForest</em> (building the model and validate, with test instances) and <em>weka.filters.supervised.attribute.AddClassification</em> (loading the model and predict, with same instances) , theoretically, be the same?</p>
  
  <p>My training code:</p>
  
  <pre><code>java -Xmx6000m -classpath "C:\Program Files\Weka-3-6\weka.jar" weka.classifiers.trees.RandomForest ^
  -t xx\trainingset.arff ^
  -T xx\testset.arff ^
  -d xx\modelfile.model ^
  -i ^
   &gt; xx\output.txt
  </code></pre>
  
  <p>My prediction code:</p>
  
  <pre><code>java -Xmx6000m -classpath "C:\Program Files\Weka-3-6\weka.jar" weka.filters.supervised.attribute.AddClassification ^
  -serialized xx\modelfile.model ^
  -classification ^
  -i xx\arfffile_tile1.arff ^
  -remove-old-class ^
  -o xx\classified_tile1.arff ^
  -c last ^
  -distribution
  </code></pre>
  
  <p>I hope I didn't confuse you... Concerning my last question, I'm currently preparing data for a test run and I will let you know about it, but this will take a while... any ideas or suggestions are welcome :) Maybe I also got totally confused about this whole Weka stuff and miss some obvious mistake?</p>