Update README.md

update README files

README.md

@@ -13,24 +13,22 @@ metrics:
 pipeline_tag: question-answering
 ---
 # Model Card for ViT5-base fine-tuned model for question answering task
-
 ## Overview
 ViT5 is a pretrained text-to-text transformer model designed specifically for Vietnamese language generation tasks. It is based on the T5 (Text-to-Text Transfer Transformer) architecture developed by Google, which has been adapted and fine-tuned for the Vietnamese language. ViT5 is capable of handling various natural language processing (NLP) tasks such as translation, summarization, question answering, and text generation, all within the Vietnamese linguistic context.
-
 ## Question answering view
-  1. Task Formulation
+  1. **Task Formulation**<br>
 In the text-to-text framework, the question answering task is formulated as "Answer the question: [question] Context: [context]".
 The input consists of a question and a related context (a passage or document) that contains the information needed to answer the question.
-  2. Input Processing
+  3. **Input Processing**<br>
 Tokenization: The combined question and context are tokenized into subword units using ViT5's tokenizer, which is pretrained for Vietnamese.
 Task Specification: The input is prefixed with a task-specific instruction to help the model understand the nature of the task.
-  3. Encoding
+  4. **Encoding**<br>
 Embedding: The tokenized input is converted into embeddings.
 Self-Attention: The encoder applies self-attention mechanisms to generate context-aware representations of the input text, integrating information from both the question and the context.
-  4. Decoding
+  5. **Decoding**<br>
 Conditional Generation: The decoder generates the output text (the answer) based on the encoded representations. The cross-attention mechanism helps the decoder focus on relevant parts of the context while generating the answer.
 Output Tokenization: The generated tokens are converted back into human-readable text (the answer).
-5. Post-Processing
+  6. **Post-Processing**<br>
 Detokenization: The output tokens are detokenized to form a coherent and fluent answer.
 Answer Extraction: The model's output is refined to ensure that the generated text is a precise and relevant answer to the input question.
 
@@ -52,24 +50,30 @@ The Vietnamese QA dataset, created by Nguyen et al. (2020), is known as UIT-ViQu
   - plausible_answer: For unanswerable questions, this provides a seemingly correct but actually incorrect answer extracted from the context.
 
 # The term for hyperparameters used in the fine-tuning process
-  - epochs = 4
-  - batch_size = 16
-  - learning rate = 2e-5
-  - evaluation strategy = "steps"
-  - save_total_limit = 1
-  - save_steps = 2000
-  - eval_steps = 2000
-  - gradient_accumulation_steps = 2
-  - eval_accumulation_steps = 2
-  - load_best_model_at_end = True
-
-# Best result
-  - epoch = 3250207813798838
-  - grad_norm = 136582374572754
-  - learning_rate = 3.3610648918469217e-06
-  - loss = 0.9397
-  - step = 2000
-  - eval_loss = 0.7907648682594299
+  - epochs: 4
+  - batch_size: 16
+  - learning rate: 2e-5
+  - evaluation strategy: "steps"
+  - save_total_limit: 1
+  - save_steps: 2000
+  - eval_steps: 2000
+  - gradient_accumulation_steps: 2
+  - eval_accumulation_steps: 2
+  - load_best_model_at_end: True
+
+# Best model saved while tuning
+  - epoch: 3.3264033264033266
+  - learning_rate: 3.3679833679833685e-06
+  - train_loss: 0.4473
+  - eval_loss: 1.2475123405456543
+
+# Evaluation
+  - validation:
+    - F1-score: 75.4081
+    - Exact-match: 58.6788
+  - test:
+    - F1-score: 78.646
+    - Exact-match: 59.147
 
 # Inference
 ## Using a pipeline as a high-level helper
@@ -84,7 +88,7 @@ question="""
 Trường UIT mang trong mình nhiệm vụ gì?
 """
 
-pipe = pipeline("question-answering", model="PhucDanh/Bartpho-fine-tuning-model-for-question-answering")
+pipe = pipeline("question-answering", model="PhucDanh/vit5-fine-tuning-for-question-answering")
 pipe(question=question, context=context)
 ```
 
@@ -102,12 +106,15 @@ question="""
 Trường UIT mang trong mình nhiệm vụ gì?
 """
 
-tokenizer = AutoTokenizer.from_pretrained("PhucDanh/Bartpho-fine-tuning-model-for-question-answering")
-tokenizer.model_input_names.remove("token_type_ids")
+tokenizer = AutoTokenizer.from_pretrained("PhucDanh/vit5-fine-tuning-for-question-answering")
+try:
+  tokenizer.model_input_names.remove("token_type_ids")
+except:
+  print("already removed!!!")
 
 inputs = tokenizer(question, context, return_tensors="pt")
 
-model = AutoModelForQuestionAnswering.from_pretrained("PhucDanh/Bartpho-fine-tuning-model-for-question-answering")
+model = AutoModelForQuestionAnswering.from_pretrained("PhucDanh/vit5-fine-tuning-for-question-answering")
 with torch.no_grad():
     outputs = model(**inputs)
 
@@ -123,7 +130,7 @@ Contact for API token authentication
 ```py
 import requests
 
-API_URL = "https://api-inference.huggingface.co/models/PhucDanh/Bartpho-fine-tuning-model-for-question-answering"
+API_URL = "https://api-inference.huggingface.co/models/PhucDanh/vit5-fine-tuning-for-question-answering"
 headers = {"Authorization": "Bearer hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
 
 def query(payload):
@@ -132,20 +139,30 @@ def query(payload):
 	
 output = query({
 	"inputs": {
-		"question": "What is my name?",
-		"context": "My name is Clara and I live in Berkeley."
-	},
+	"question": "What is my name?",
+	"context": "My name is Clara and I live in Berkeley."
+  },
 })
 ```
 
 # Reference
 ## Model:
 ```
-@article{tran2021bartpho,
-  title={BartPho: pre-trained sequence-to-sequence models for Vietnamese},
-  author={Tran, Nguyen Luong and Le, Duong Minh and Nguyen, Dat Quoc},
-  journal={arXiv preprint arXiv:2109.09701},
-  year={2021}
+@article{phan2022vit5,
+  title={Vit5: Pretrained text-to-text transformer for vietnamese language generation},
+  author={Phan, Long and Tran, Hieu and Nguyen, Hieu and Trinh, Trieu H},
+  journal={arXiv preprint arXiv:2205.06457},
+  year={2022}
+}
+
+@article{raffel2020exploring,
+  title={Exploring the limits of transfer learning with a unified text-to-text transformer},
+  author={Raffel, Colin and Shazeer, Noam and Roberts, Adam and Lee, Katherine and Narang, Sharan and Matena, Michael and Zhou, Yanqi and Li, Wei and Liu, Peter J},
+  journal={Journal of machine learning research},
+  volume={21},
+  number={140},
+  pages={1--67},
+  year={2020}
 }
 ```
 ## Dataset: