Update README.md
Browse files
README.md
CHANGED
@@ -31,24 +31,6 @@ This is the model card for **phi-2-dialogsum**, a dialogue summarization model b
|
|
31 |
### Direct Use
|
32 |
This model can be used directly for **dialogue summarization** tasks. For example, given a multi-turn conversation, the model will produce a succinct summary capturing the key information and context.
|
33 |
|
34 |
-
### Downstream Use [optional]
|
35 |
-
Could be fine-tuned or adapted for other text summarization tasks, especially conversation-like data (customer service transcripts, chat logs, interviews, etc.).
|
36 |
-
|
37 |
-
### Out-of-Scope Use
|
38 |
-
- Generating harmful or misleading content.
|
39 |
-
- Deploying in high-stakes scenarios without proper validation (e.g., medical or legal advice).
|
40 |
-
|
41 |
-
## Bias, Risks, and Limitations
|
42 |
-
|
43 |
-
- **Biases:** The model may reflect biases present in the data used to train or fine-tune it.
|
44 |
-
- **Risks:** Summaries could omit critical context or misrepresent the conversation.
|
45 |
-
- **Limitations:** The model’s performance may degrade on conversations with specialized jargon, code-switching, or extremely long contexts.
|
46 |
-
|
47 |
-
### Recommendations
|
48 |
-
- Always review generated summaries for accuracy.
|
49 |
-
- Be mindful of potential biases or omissions.
|
50 |
-
- Avoid using the model as the sole source of truth in sensitive domains.
|
51 |
-
|
52 |
## How to Get Started with the Model
|
53 |
|
54 |
Below is a quick code snippet to load and run inference with this model:
|
@@ -69,6 +51,25 @@ inputs = tokenizer([input_text], max_length=512, truncation=True, return_tensors
|
|
69 |
summary_ids = model.generate(**inputs, max_length=60, num_beams=4, early_stopping=True)
|
70 |
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
|
71 |
|
72 |
-
print("Summary:", summary)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
73 |
|
|
|
|
|
|
|
|
|
74 |
|
|
|
31 |
### Direct Use
|
32 |
This model can be used directly for **dialogue summarization** tasks. For example, given a multi-turn conversation, the model will produce a succinct summary capturing the key information and context.
|
33 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
34 |
## How to Get Started with the Model
|
35 |
|
36 |
Below is a quick code snippet to load and run inference with this model:
|
|
|
51 |
summary_ids = model.generate(**inputs, max_length=60, num_beams=4, early_stopping=True)
|
52 |
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
|
53 |
|
54 |
+
print("Summary:", summary)
|
55 |
+
```
|
56 |
+
|
57 |
+
## Training Details
|
58 |
+
|
59 |
+
Training dataset (Dialogsum)[https://huggingface.co/datasets/neil-code/dialogsum-test]
|
60 |
+
|
61 |
+
## Evaluation
|
62 |
+
|
63 |
+
ORIGINAL MODEL:
|
64 |
+
{'rouge1': 0.2990526195120211, 'rouge2': 0.10874019046839419, 'rougeL': 0.21186900909813286, 'rougeLsum': 0.22342464591439556}
|
65 |
+
|
66 |
+
PEFT MODEL:
|
67 |
+
{'rouge1': 0.3132817683433486, 'rouge2': 0.1070363134080079, 'rougeL': 0.23226760188839027, 'rougeLsum': 0.25947902747914586}
|
68 |
+
|
69 |
+
## Absolute percentage improvement of PEFT MODEL over ORIGINAL MODEL
|
70 |
|
71 |
+
rouge1: 1.42%
|
72 |
+
rouge2: -0.17%
|
73 |
+
rougeL: 2.04%
|
74 |
+
rougeLsum: 3.61%
|
75 |
|