title: How to compare sentence similarities using embeddings from BERT
body: <p>I am using the HuggingFace Transformers package to access pretrained models. As my use case needs functionality for both English and Arabic, I am using the <a href="https://github.com/google-research/bert/blob/master/multilingual.md" rel="noreferrer">bert-base-multilingual-cased</a> pretrained model. I need to be able to compare the similarity of sentences using something such as cosine similarity. To use  this, I first need to get an embedding vector for each sentence, and can then compute the cosine similarity.</p>
  
  <p>Firstly, what is the best way to extratc the semantic embedding from the BERT model? Would taking the last hidden state of the model after being fed the sentence suffice?</p>
  
  <pre><code>import torch
  from transformers import BertModel, BertTokenizer
  
  model_class = BertModel
  tokenizer_class = BertTokenizer
  pretrained_weights = 'bert-base-multilingual-cased'
  
  tokenizer = tokenizer_class.from_pretrained(pretrained_weights)
  model = model_class.from_pretrained(pretrained_weights)
  
  sentence = 'this is a test sentence'
  
  input_ids = torch.tensor([tokenizer.encode(sentence, add_special_tokens=True)])
  with torch.no_grad():
      output_tuple = model(input_ids)
      last_hidden_states = output_tuple[0]
  
  print(last_hidden_states.size(), last_hidden_states)
  </code></pre>
  
  <p>Secondly, if this is a sufficient way to get embeddings from my sentence, I now have another problem where the embedding vectors have different lengths depending on the length of the original sentence. The shapes output are <code>[1, n, vocab_size]</code>, where <code>n</code> can have any value. </p>
  
  <p>In order to compute two vectors' cosine similarity, they need to be the same  length. How can I do this here? Could something as naive as first summing across <code>axis=1</code> still work? What other options do I have? </p>