title: OpenNLP-Document Categorizer- how to classify documents based on status; language of docs not English, also default features?
body: <p>I want to classify my documents using OpenNLP's Document Categorizer, based on their status: pre-opened, opened, locked, closed etc.</p>
  
  <p>I have 5 classes and I'm using the Naive Bayes algorithm, 60 documents in my training set, and trained my set on 1000 iterations with 1 cut off param.</p>
  
  <p>But no success, when I test them I don't get good results. I was thinking maybe it is because of the language of the documents (is not in English) or maybe I should somehow add the statuses as features. I have set the default features in the categorizer, and also I'm not very familiar with them.</p>
  
  <p>The result should be locked, but its categorized as opened.</p>
  
  <pre><code>InputStreamFactory in=null;
  try {
  in= new MarkableFileInputStreamFactory(new 
  File("D:\\JavaNlp\\doccategorizer\\doccategorizer.txt"));
  }
  catch (FileNotFoundException e2) {
  System.out.println("Creating new input stream");
  e2.printStackTrace();
  }
  
  ObjectStream lineStream=null;
  ObjectStream sampleStream=null;
  
  try {
  lineStream = new PlainTextByLineStream(in, "UTF-8");
  sampleStream = new DocumentSampleStream(lineStream);            
  }
  catch (IOException e1) {
  System.out.println("Document Sample Stream");
  e1.printStackTrace();
  }
  
  
  TrainingParameters params = new TrainingParameters();
  params.put(TrainingParameters.ITERATIONS_PARAM, 1000+"");
  params.put(TrainingParameters.CUTOFF_PARAM, 1+"");
  params.put(AbstractTrainer.ALGORITHM_PARAM, 
  NaiveBayesTrainer.NAIVE_BAYES_VALUE);
  
  
  DoccatModel model=null;
  try {
  model = DocumentCategorizerME.train("en", sampleStream, params, new 
  DoccatFactory());
  } 
  catch (IOException e) 
  {
  System.out.println("Training...");
  e.printStackTrace();
  }
  
  
  System.out.println("\nModel is successfully trained.");
  
  
  BufferedOutputStream modelOut=null;
  
  try {
  modelOut = new BufferedOutputStream(new 
  FileOutputStream("D:\\JavaNlp\\doccategorizer\\classifier-maxent.bin"));
  } 
  catch (FileNotFoundException e) {
  
  System.out.println("Creating output stream");
  e.printStackTrace();
  }
  try {
  model.serialize(modelOut);
  }
  catch (IOException e) {
  
  System.out.println("Serialize...");
  e.printStackTrace();
  }
  System.out.println("\nTrained model is kept in: 
  "+"model"+File.separator+"en-cases-classifier-maxent.bin");
  
  DocumentCategorizer doccat = new DocumentCategorizerME(model);
  String[] docWords = "Some text here...".replaceAll("[^A-Za-z]", " ").split(" ");
  double[] aProbs = doccat.categorize(docWords);
  
  
  System.out.println("\n---------------------------------\nCategory : 
  Probability\n---------------------------------");
  for(int i=0;i&lt;doccat.getNumberOfCategories();i++){
  System.out.println(doccat.getCategory(i)+" : "+aProbs[i]);
  }
  System.out.println("---------------------------------");
  
  System.out.println("\n"+doccat.getBestCategory(aProbs)+" : is the category 
  for the given sentence");
  </code></pre>
  
  <p><img src="https://photos.app.goo.gl/GEgrCSJjk8NqaSe7A" alt="results"> 
  <img src="https://photos.app.goo.gl/j2qDeLNm6wiw9nzx8" alt="results2"></p>
  
  <p>Can someone make a suggestion for me how to categorize my documents well, like should I add a language detector first, or add new features?</p>
  
  <p>Thanks in advance</p>