title: How to apply word2vec for k-means clustering?
body: <p>I am new to word2vec. With applying this method, I am trying to form some clusters  based on words extracted by word2vec from scientific publications' abstracts. To this end, I have first retrieved sentences from the abstracts via stanfordNLP and put each sentence into a line in a text file. Then the text file required by deeplearning4j word2vec was ready to process (<a href="http://deeplearning4j.org/word2vec" rel="nofollow">http://deeplearning4j.org/word2vec</a>).</p>
  
  <p>Since the texts come from scientific fields, there are a lot of mathematical terms or brackets. See the sample sentences below:</p>
  
  <pre><code>The meta-analysis showed statistically significant effects of pharmacopuncture compared to conventional treatment = 3.55 , P = .31 , I-2 = 16 % ) . 
  
  90 asymptomatic hypertensive subjects associated with LVH , DM , or RI were randomized to receive D&amp;G herbal capsules 1 gm/day , 2 gm/day , or identical placebo capsules in double-blind and parallel fashion for 12 months . 
  </code></pre>
  
  <p>After preparing the text file, I have run word2vec as below:</p>
  
  <pre><code>SentenceIterator iter = new LineSentenceIterator(new File(".../filename.txt"));
          iter.setPreProcessor(new SentencePreProcessor() {
              @Override
              public String preProcess(String sentence) {
                  //System.out.println(sentence.toLowerCase());
                  return sentence.toLowerCase();
              }
          });
  
  
          // Split on white spaces in the line to get words
          TokenizerFactory t = new DefaultTokenizerFactory();
          t.setTokenPreProcessor(new CommonPreprocessor());
  
          log.info("Building model....");
          Word2Vec vec = new Word2Vec.Builder()
                  .minWordFrequency(5)
                  .iterations(1)
                  .layerSize(100)
                  .seed(42)
                  .windowSize(5)
                  .iterate(iter)
                  .tokenizerFactory(t)
                  .build();
  
          log.info("Fitting Word2Vec model....");
          vec.fit();
  
          log.info("Writing word vectors to text file....");
  
          // Write word vectors
          WordVectorSerializer.writeWordVectors(vec, "abs_terms.txt");
  </code></pre>
  
  <p>This script creates a text file containing many words withe their related vector values in each row as below:</p>
  
  <pre><code>pills -4.559159278869629E-4 0.028691953048110008 0.023867368698120117 ...
  tricuspidata -0.00431067543104291 -0.012515762820839882 0.0074045853689312935 ...
  </code></pre>
  
  <p>As a subsequent step, this text file has been used to form some clusters via k-means in spark. See the code below:</p>
  
  <pre><code>    val rawData = sc.textFile("...abs_terms.txt")
      val extractedFeatureVector = rawData.map(s =&gt; Vectors.dense(s.split(' ').slice(2,101).map(_.toDouble))).cache()
  
      val numberOfClusters = 10
      val numberOfInterations = 100
  
      //We use KMeans object provided by MLLib to run
      val modell = KMeans.train(extractedFeatureVector, numberOfClusters, numberOfInterations)
  
      modell.clusterCenters.foreach(println)
  
  
      //Get cluster index for each buyer Id
      val AltCompByCluster = rawData.map {
        row=&gt;
          (modell.predict(Vectors.dense(row.split(' ').slice(2,101)
            .map(_.toDouble))),row.split(',').slice(0,1).head)
      }
  
      AltCompByCluster.foreach(println)
  </code></pre>
  
  <p>As a result of the latest scala code above, I have retrieved 10 clusters based on the word vectors suggested by word2vec. However, when I have checked my clusters no obvious common words appeared. That is, I could not get reasonable clusters as I expected. Based on this bottleneck of mine I have a few questions:</p>
  
  <p>1) From some tutorials for word2vec I have seen that no data cleaning is made. In other words, prepositions etc. are left in the text. So how should I apply cleaning procedure when applying word2vec?</p>
  
  <p>2) How can I visualize the clustering results in a explanatory way?</p>
  
  <p>3) Can I use word2vec word vectors as input to neural networks? If so which neural network (convolutional, recursive, recurrent) method would be more suitable for my goal? </p>
  
  <p>4) Is word2vec meaningful for my goal?</p>
  
  <p>Thanks in advance.</p>