title: Confusion in Pre-processing text for Roberta Model
body: <p>I want to apply Roberta model for  text similarity. Given a pair of sentences,the input should be in the format <code>&lt;s&gt; A &lt;/s&gt;&lt;/s&gt; B &lt;/s&gt;</code>. I figure out two possible ways to generate the input ids namely</p>
  
  <p>a)</p>
  
  <pre class="lang-py prettyprint-override"><code>from transformers import AutoTokenizer, AutoModel
  
  tokenizer = AutoTokenizer.from_pretrained('roberta-base')
  
  list1 = tokenizer.encode('Very severe pain in hands')
  
  list2 = tokenizer.encode('Numbness of upper limb')
  
  sequence = list1+[2]+list2[1:]
  
  </code></pre>
  
  <p>In this case, sequence is <code>[0, 12178, 3814, 2400, 11, 1420, 2, 2, 234, 4179, 1825, 9, 2853, 29654, 2]</code></p>
  
  <p>b)</p>
  
  <pre class="lang-py prettyprint-override"><code>from transformers import AutoTokenizer, AutoModel
  
  tokenizer = AutoTokenizer.from_pretrained('roberta-base')
  
  list1 = tokenizer.encode('Very severe pain in hands', add_special_tokens=False)
  
  list2 = tokenizer.encode('Numbness of upper limb', add_special_tokens=False)
  
  sequence = [0]+list1+[2,2]+list2+[2]
  </code></pre>
  
  <p>In this case, sequence is <code>[0, 25101, 3814, 2400, 11, 1420, 2, 2, 487, 4179, 1825, 9, 2853, 29654, 2]</code></p>
  
  <p>Here <code>0</code> represents <code>&lt;s&gt;</code> token and 2 represents <code>&lt;/s&gt;</code> token. I'm not sure which is the correct way to encode the given two sentences for calculating sentence similarity using Roberta model.</p>