uer commited on
Commit
057298c
·
1 Parent(s): ef820fc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -1
README.md CHANGED
@@ -96,7 +96,7 @@ output = model(encoded_input)
96
 
97
  ## Training data
98
 
99
- CLUECorpusSmall is used as training data. We found that models pre-trained on CLUECorpusSmall outperform those pre-trained on CLUECorpus2020, although CLUECorpus2020 is much larger than CLUECorpusSmall.
100
 
101
  ## Training procedure
102
 
@@ -142,6 +142,13 @@ python3 pretrain.py --dataset_path cluecorpussmall_seq512_dataset.pt \
142
  --tie_weights --embedding word_pos_seg --encoder transformer --mask fully_visible --target mlm
143
  ```
144
 
 
 
 
 
 
 
 
145
  ### BibTeX entry and citation info
146
 
147
  ```
 
96
 
97
  ## Training data
98
 
99
+ [CLUECorpusSmall](https://github.com/CLUEbenchmark/CLUECorpus2020/) is used as training data. We found that models pre-trained on CLUECorpusSmall outperform those pre-trained on CLUECorpus2020, although CLUECorpus2020 is much larger than CLUECorpusSmall.
100
 
101
  ## Training procedure
102
 
 
142
  --tie_weights --embedding word_pos_seg --encoder transformer --mask fully_visible --target mlm
143
  ```
144
 
145
+ Finally, we convert the pre-trained model into Huggngface's format:
146
+ ```
147
+ python3 scripts/convert_bert_from_uer_to_huggingface.py --input_model_path pytorch_model.bin
148
+ --output_model_path huggingface_model.bin
149
+ --layers_num 12 --target mlm
150
+ ```
151
+
152
  ### BibTeX entry and citation info
153
 
154
  ```