title: How is the semantic similarity score calculated in STS Benchmark dataset?
body: <p>This is the GitHub repo : <a href="https://github.com/brmson/dataset-sts" rel="nofollow noreferrer">https://github.com/brmson/dataset-sts</a></p>
  
  <p>The STS Benchmark dataset contains about 4000 pairs of similar and dissimilar  sentences along with their semantic similarity scores.</p>
  
  <p>Task that I'm trying to do: 
  I have another custom dataset which also has pairs of similar and dissimilar sentences. ( with just 200 pairs) </p>
  
  <p>I want to combine these two datasets (STS &amp; my custom dataset) and use that for fine tuning a Bert model. 
  (Bert sentence transformer: <a href="https://github.com/UKPLab/sentence-transformers" rel="nofollow noreferrer">https://github.com/UKPLab/sentence-transformers</a>)</p>
  
  <p>But the model needs the semantic similarity score of all the pairs of sentences. How do I compute that score for the sentences that I have in my custom dataset? </p>
  
  <p>It has to be computed in the same way as how it was computed for the pairs of sentences in the STS Benchmark dataset.</p>
  
  <p>This thread is very similar but it didn't quite answer the question that I am looking for : <a href="https://stackoverflow.com/questions/59172532/bert-fine-tuned-for-semantic-similarity">Bert fine-tuned for semantic similarity</a></p>