MinistryofDigitalAffairs commited on
Commit
1dc9bf7
·
verified ·
1 Parent(s): cc0b143

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -52,7 +52,7 @@ Below is a summary of the main PLLuM models, including their licenses, bases, an
52
 
53
  ### Model Development
54
  - **Pretraining**: All models were pretrained or continued-pretrained on large-scale Polish corpora (up to 150B tokens) plus a range of additional Slavic/Baltic and English texts.
55
- - **Instruction Fine-Tuning**: We refined the models on manually curated Polish “organic instructions,” converted instructions from premium Polish corpora, and synthetic instructions generated by strong LLMs.
56
  - **Alignment and Preference Learning**: Manually annotated preference data taught the models to produce safer, balanced, and contextually appropriate responses, even in adversarial or sensitive cases.
57
  - **Domain-Specific Adaptations**: Specialized RAG-based (Retrieval Augmented Generation) models were developed for tasks like public administration, demonstrating strong performance in complex information retrieval and question answering.
58
 
 
52
 
53
  ### Model Development
54
  - **Pretraining**: All models were pretrained or continued-pretrained on large-scale Polish corpora (up to 150B tokens) plus a range of additional Slavic/Baltic and English texts.
55
+ - **Instruction Fine-Tuning**: We refined the models on manually curated Polish “organic instructions (approx. 40k), converted instructions from premium Polish corpora (approx. 50k), and synthetic instructions generated by strong LLMs (approx. 10k).
56
  - **Alignment and Preference Learning**: Manually annotated preference data taught the models to produce safer, balanced, and contextually appropriate responses, even in adversarial or sensitive cases.
57
  - **Domain-Specific Adaptations**: Specialized RAG-based (Retrieval Augmented Generation) models were developed for tasks like public administration, demonstrating strong performance in complex information retrieval and question answering.
58