<p>Have you tried the <a href="https://www.aclweb.org/anthology/D18-2029.pdf" rel="nofollow noreferrer">Universal Sentence Encoder (USE)</a>, or the <a href="https://arxiv.org/abs/1907.04307" rel="nofollow noreferrer">Multilingual Universal Sentence Encoder</a>? </p>

<p>There's a colab showing how to score sentence pairs for <a href="https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/semantic_similarity_with_tf_hub_universal_encoder.ipynb" rel="nofollow noreferrer">semantic textual similarity with USE</a> on the Semantic Textual Similarity Benchmark (STS-B) and another for <a href="https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/cross_lingual_similarity_with_tf_hub_multilingual_universal_encoder.ipynb" rel="nofollow noreferrer">multilingual similarity</a>.</p>

<p>Here's a heatmap of pairwise semantic similarity scores from USE on the <a href="https://ai.googleblog.com/2018/05/advances-in-semantic-textual-similarity.html" rel="nofollow noreferrer">Google AI blog post Advances in Semantic Textual Similarity</a>. The model was trained on a large amount of web data so it should work well for a wide variety of input data.</p>

<p><img src="https://i.stack.imgur.com/bJMue.png?s=512" alt="Pairwise semantic similarity comparison via outputs from TensorFlow Hub Universal Sentence Encoder."> </p>
How to find Sentence Similarity using deep learning?
<p>I am trying to find sentence similarity through word emebeddings and then applying cosine similarity score. Tried CBOW/Skip Gram methods for embedding but did not solve the problem.</p>

<p>I am doing this for product review data. I have two columns:</p>

<pre><code>SNo         Product_Title                                Customer_Review   
 1       101.x battery works well                    I have an Apple phone and it's not that
          with Samsung smart phone                     that great.

 2       112.x battery works well                     I have samsung smart tv and I tell that it's
         with Samsung smart phone                     not wort buying.

 3      112.x battery works well                      This charger works very well with samsung 
        with Samsung smart phone.                      phone. It is fast charging.
</code></pre>

<p>The first two reviews are <code>irrelevant</code> as semantic meaning of <code>Product_Title</code> and <code>Customer_Review</code>  are completely different.</p>

<p>How can an algorithm find this semantic meaning of sentences and score them.</p>

<p>My Approach:</p>

<ol>
<li><p>Text pre-processing</p></li>
<li><p>Train CBOW/Skip gram using Gensim on my data-set</p></li>
<li><p>Do Sentence level encoding via averaging all word vectors in that sentence </p></li>
<li><p>Take cosine similarity of <code>product_title</code> and <code>reviews</code>.</p></li>
</ol>

<p>Problem: It was not able to find the context from the sentence and hence the result was very poor.</p>

<p>Approch 2:</p>

<p>Used pre-trained BERT without pre-processing sentences. The result was not improving either.</p>

<p>1.Any other approach that would capture the context/semantics of sentences.</p>

<p>2.How can we train BERT on our data-set from scratch without using pre-trained model?</p>