Update README.md
Browse files
README.md
CHANGED
@@ -52,7 +52,7 @@ Below is a summary of the main PLLuM models, including their licenses, bases, an
|
|
52 |
|
53 |
### Model Development
|
54 |
- **Pretraining**: All models were pretrained or continued-pretrained on large-scale Polish corpora (up to 150B tokens) plus a range of additional Slavic/Baltic and English texts.
|
55 |
-
- **Instruction Fine-Tuning**: We refined the models on manually curated Polish “organic instructions
|
56 |
- **Alignment and Preference Learning**: Manually annotated preference data taught the models to produce safer, balanced, and contextually appropriate responses, even in adversarial or sensitive cases.
|
57 |
- **Domain-Specific Adaptations**: Specialized RAG-based (Retrieval Augmented Generation) models were developed for tasks like public administration, demonstrating strong performance in complex information retrieval and question answering.
|
58 |
|
|
|
52 |
|
53 |
### Model Development
|
54 |
- **Pretraining**: All models were pretrained or continued-pretrained on large-scale Polish corpora (up to 150B tokens) plus a range of additional Slavic/Baltic and English texts.
|
55 |
+
- **Instruction Fine-Tuning**: We refined the models on manually curated Polish “organic instructions” (approx. 40k), converted instructions from premium Polish corpora (approx. 50k), and synthetic instructions generated by strong LLMs (approx. 10k).
|
56 |
- **Alignment and Preference Learning**: Manually annotated preference data taught the models to produce safer, balanced, and contextually appropriate responses, even in adversarial or sensitive cases.
|
57 |
- **Domain-Specific Adaptations**: Specialized RAG-based (Retrieval Augmented Generation) models were developed for tasks like public administration, demonstrating strong performance in complex information retrieval and question answering.
|
58 |
|