Training in progress epoch 0

.ipynb_checkpoints/README-checkpoint.md

@@ -0,0 +1,75 @@
+---
+license: apache-2.0
+tags:
+- Question Answering
+metrics:
+- squad
+model-index:
+- name: question-answering-roberta-base-s
+  results: []
+---
+
+# Question Answering 
+The model is intended to be used for Q&A task, given the question & context, the model would attempt to infer the answer text, answer span & confidence score.<br>
+Model is encoder-only (roberta-base) with QuestionAnswering LM Head, fine-tuned on SQUADx dataset with **exact_match:** 86.14 & **f1:** 92.330 performance scores.
+
+[Live Demo: Question Answering Encoders vs Generative](https://huggingface.co/spaces/anshoomehra/question_answering)
+
+Please follow this link for [Generative Question Answering](https://huggingface.co/anshoomehra/question-answering-generative-t5-v1-base-s-q-c/)
+
+Example code:
+```
+from transformers import pipeline
+
+model_checkpoint = "anshoomehra/question-answering-roberta-base-s"
+
+context = """
+🤗 Transformers is backed by the three most popular deep learning libraries — Jax, PyTorch and TensorFlow — with a seamless integration
+between them. It's straightforward to train your models with one before loading them for inference with the other.
+"""
+question = "Which deep learning libraries back 🤗 Transformers?"
+
+question_answerer = pipeline("question-answering", model=model_checkpoint)
+question_answerer(question=question, context=context)
+
+```
+
+## Training and evaluation data
+
+SQUAD Split
+
+## Training procedure
+
+Preprocessing:
+1. SQUAD Data longer chunks were sub-chunked with input context max-length 384 tokens and stride as 128 tokens.
+2. Target answers readjusted for sub-chunks, sub-chunks with no-answers or partial answers were set to target answer span as (0,0)
+
+Metrics:
+1. Adjusted accordingly to handle sub-chunking.
+2. n best = 20
+3. skip answers with length zero or higher than max answer length (30)
+
+### Training hyperparameters
+Custom Training Loop:
+The following hyperparameters were used during training:
+- learning_rate: 2e-5
+- train_batch_size: 32
+- eval_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 2
+
+### Training results
+
+| Epoch | F1       | Exact Match | 
+|:-----:|:--------:|:-----------:|
+| 1.0   | 91.3085  | 84.5412     |
+| 2.0   | 92.3304  | 86.1400     |
+
+
+### Framework versions
+
+- Transformers 4.23.0.dev0
+- Pytorch 1.12.1+cu113
+- Datasets 2.5.2
+- Tokenizers 0.13.0

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a8514a839a8973804c295867152ab37e4eda1435cf81aa80fc2cc4fac6e231e6
+oid sha256:7b09dfc116156233cf0f52b50b0a6d8b59edcc0860ec693be6e489c896701200
 size 496297393