Update README.md

README.md

@@ -1,3 +1,100 @@
 ---
 license: mit
+language: en
+tags:
+- bert
+- cloze
+- distractor
+- generation
+datasets:
+- dgen
+widget:
+- text: "The only known planet with large amounts of water is [MASK]. [SEP] earth"
+- text: "The products of photosynthesis are glucose and [MASK] else. [SEP] oxygen"
 ---
+
+# cdgp-csg-scibert-dgen
+
+## Model description
+
+This model is a Candidate Set Generator in **"CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language Model", Findings of EMNLP 2022**.
+
+Its input are stem and answer, and output is candidate set of distractors. It is fine-tuned by [**DGen**](https://github.com/DRSY/DGen) dataset based on [**allenai/scibert_scivocab_uncased**](https://huggingface.co/allenai/scibert_scivocab_uncased) model.
+
+For more details, you can see our **paper** or [**GitHub**](https://github.com/AndyChiangSH/CDGP).
+
+## How to use?
+
+1. Download model by hugging face transformers.
+```python
+from transformers import BertTokenizer, BertForMaskedLM, pipeline
+
+tokenizer = BertTokenizer.from_pretrained("AndyChiang/cdgp-csg-scibert-dgen")
+csg_model = BertForMaskedLM.from_pretrained("AndyChiang/cdgp-csg-scibert-dgen")
+```
+
+2. Create a unmasker.
+```python
+unmasker = pipeline("fill-mask", tokenizer=tokenizer, model=csg_model, top_k=10)
+```
+
+3. Use the unmasker to generate the candidate set of distractors.
+```python
+sent = "The only known planet with large amounts of water is [MASK]. [SEP] earth"
+cs = unmasker(sent)
+print(cs)
+```
+
+## Dataset
+
+This model is fine-tuned by [DGen](https://github.com/DRSY/DGen) dataset, which covers multiple domains including science, vocabulary, common sense and trivia. It is compiled from a wide variety of datasets including SciQ, MCQL, AI2 Science Questions, etc. The detail of DGen dataset is shown below.
+
+| DGen dataset            | Train | Valid | Test | Total |
+| ----------------------- | ----- | ----- | ---- | ----- |
+| **Number of questions** | 2321  | 300   | 259  | 2880  |
+
+You can also use the [dataset](https://github.com/AndyChiangSH/CDGP/blob/main/datasets/DGen.zip) we have already cleaned.
+
+## Training
+
+We use a special way to fine-tune model, which is called **"Answer-Relating Fine-Tune"**. More details are in our paper.
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+
+- Pre-train language model: [allenai/scibert_scivocab_uncased](https://huggingface.co/allenai/scibert_scivocab_uncased) 
+- Optimizer: adam
+- Learning rate: 0.0001
+- Max length of input: 64
+- Batch size: 64
+- Epoch: 1
+- Device: NVIDIA® Tesla T4 in Google Colab 
+
+## Testing
+
+The evaluations of this model as a Candidate Set Generator in CDGP is as follows:
+
+| P@1   | F1@3  | MRR   | NDCG@10 |
+| ----- | ----- | ----- | ------- |
+| 13.13 | 12.23 | 25.12 | 34.17   |
+
+## Other models
+
+### Candidate Set Generator
+
+| Models      | CLOTH                                                                               | DGen                                                                             |
+| ----------- | ----------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
+| **BERT**    | [cdgp-csg-bert-cloth](https://huggingface.co/AndyChiang/cdgp-csg-bert-cloth)        | [cdgp-csg-bert-dgen](https://huggingface.co/AndyChiang/cdgp-csg-bert-dgen)       |
+| **SciBERT** | [cdgp-csg-scibert-cloth](https://huggingface.co/AndyChiang/cdgp-csg-scibert-cloth)  | [*cdgp-csg-scibert-dgen*](https://huggingface.co/AndyChiang/cdgp-csg-scibert-dgen) |
+| **RoBERTa** | [cdgp-csg-roberta-cloth](https://huggingface.co/AndyChiang/cdgp-csg-roberta-cloth) | [cdgp-csg-roberta-dgen](https://huggingface.co/AndyChiang/cdgp-csg-roberta-dgen) |
+| **BART**    | [cdgp-csg-bart-cloth](https://huggingface.co/AndyChiang/cdgp-csg-bart-cloth)        | [cdgp-csg-bart-dgen](https://huggingface.co/AndyChiang/cdgp-csg-bart-dgen)       |
+
+### Distractor Selector
+
+**fastText**: [cdgp-ds-fasttext](https://huggingface.co/AndyChiang/cdgp-ds-fasttext)
+
+
+## Citation
+
+None