Update README.md
Browse files
README.md
CHANGED
|
@@ -15,6 +15,10 @@ Sentence Transformer for Assurance & Risk Question-Answering (STAR-QA) is a fine
|
|
| 15 |
|
| 16 |
The model was evaluated on a held-out sample from the STAR-QA dataset (see below) using `sentence-transformers.InformationRetrievalEvaluator`. Reported metrics include P/R @ 3 candidates, as well as MRR @ 10, MAP @ 10 and NDCG @ 100. This fine-tuned model was also benchmarked against its base model using the same methodology.
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
## Training Data
|
| 19 |
|
| 20 |
The model was fine-tuned from a corpus of audit, risk-management, compliance and associated regulatory documents sourced from the public internet. Documents were cleaned and chunked into 2-sentence blocks. Each block was then sent to a state-of-the-art LLM with the following prompt: "Write a question about {document_topic} for which this is the answer: {block}"
|
|
|
|
| 15 |
|
| 16 |
The model was evaluated on a held-out sample from the STAR-QA dataset (see below) using `sentence-transformers.InformationRetrievalEvaluator`. Reported metrics include P/R @ 3 candidates, as well as MRR @ 10, MAP @ 10 and NDCG @ 100. This fine-tuned model was also benchmarked against its base model using the same methodology.
|
| 17 |
|
| 18 |
+
| Model | Metric | Score |
|
| 19 |
+
|-------|--------|-------|
|
| 20 |
+
|test |test | 0.0|
|
| 21 |
+
|
| 22 |
## Training Data
|
| 23 |
|
| 24 |
The model was fine-tuned from a corpus of audit, risk-management, compliance and associated regulatory documents sourced from the public internet. Documents were cleaned and chunked into 2-sentence blocks. Each block was then sent to a state-of-the-art LLM with the following prompt: "Write a question about {document_topic} for which this is the answer: {block}"
|