uer commited on
Commit
50b9fa0
·
1 Parent(s): 7165743

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -20,7 +20,7 @@ The Text-to-Text Transfer Transformer (T5) leveraged a unified text-to-text form
20
  | **T5-Small** | [**L=6/H=512 (Small)**][small] |
21
  | **T5-Base** | [**L=12/H=768 (Base)**][base] |
22
 
23
- In T5, spans of the input sequence are masked by so-called sentinel token. Each sentinel token represents a unique mask token for the input sequence and should start with <extra_id_0>, <extra_id_1>, … up to <extra_id_199>. However, <extra_id_xxx> is separated into multiple parts in Huggingface's Hosted inference API. Therefore, we replace <extra_id_xxx> with extraxxx in vocabulary and BertTokenizer regards extraxxx as one sentinel token.
24
 
25
  ## How to use
26
 
 
20
  | **T5-Small** | [**L=6/H=512 (Small)**][small] |
21
  | **T5-Base** | [**L=12/H=768 (Base)**][base] |
22
 
23
+ In T5, spans of the input sequence are masked by so-called sentinel token. Each sentinel token represents a unique mask token for the input sequence and should start with `<extra_id_0>`, `<extra_id_1>`, … up to `<extra_id_199>`. However, `<extra_id_xxx>` is separated into multiple parts in Huggingface's Hosted inference API. Therefore, we replace `<extra_id_xxx>` with `extraxxx` in vocabulary and BertTokenizer regards `extraxxx` as one sentinel token.
24
 
25
  ## How to use
26