Update README.md

README.md

@@ -1,69 +1,70 @@
----
-license: mit
----
----
-license: mit
-datasets:
-- allenai/MADLAD-400
-language:
-- en
-- ru
-- bg
-- uk
-- kk
-base_model:
-- meta-llama/Llama-2-7b-hf
----
-VocADT is a solution for vocabulary adaptation using adapter modules that are trained to learn the optimal linear combination of existing embeddings while keeping the model’s weights fixed.
-VocADT offers a flexible and scalable solution without requiring external resources or language constraints.
-
-
-## New Vocabulary Adapted Models
-Only the input/output embeddings are replaced, while all other original weights of base model remain fixed.
-These are the merged version: after training the adapters, we merge the original embeddings with the adapter to generate the new embeddings.
-| Name | Adapted Model | Base Model | New Vocab Size | Focused Languages |
-|---|---|---|---|---|
-| VocADT-Latin-Mistral | [h-j-han/Mistral-7B-VocADT-50k-Latin](https://huggingface.co/h-j-han/Mistral-7B-VocADT-50k-Latin) | [Mistral](https://huggingface.co/mistralai/Mistral-7B-v0.1) | 50k | Swahili (sw), Indonesian (id), Estonian (et), Haitian Creole (ht), English (en)|
-| VocADT-Mixed-Mistral | [h-j-han/Mistral-7B-VocADT-50k-Mixed](https://huggingface.co/h-j-han/Mistral-7B-VocADT-50k-Mixed) | [Mistral](https://huggingface.co/mistralai/Mistral-7B-v0.1) | 50k | Korean (ko), Greek (el), Russian (ru), Bulgarian (bg), English (en) |
-| VocADT-Cyrillic-Mistral | [h-j-han/Mistral-7B-VocADT-50k-Cyrillic](https://huggingface.co/h-j-han/Mistral-7B-VocADT-50k-Cyrillic) | [Mistral](https://huggingface.co/mistralai/Mistral-7B-v0.1) | 50k | Russian (ru), Bulgarian (bg), Ukrainian (uk), Kazakh (kk), English (en) |
-|||||
-| VocADT-Latin-LLama | [h-j-han/Llama2-7B-VocADT-50k-Latin](https://huggingface.co/h-j-han/Llama2-7B-VocADT-50k-Latin) | [Llama](https://huggingface.co/meta-llama/Llama-2-7b-hf) | 50k | Swahili (sw), Indonesian (id), Estonian (et), Haitian Creole (ht), English (en)|
-| VocADT-Mixed-LLama | [h-j-han/Llama2-7B-VocADT-50k-Mixed](https://huggingface.co/h-j-han/Llama2-7B-VocADT-50k-Mixed) | [Llama](https://huggingface.co/meta-llama/Llama-2-7b-hf) | 50k | Korean (ko), Greek (el), Russian (ru), Bulgarian (bg), English (en) |
-| VocADT-Cyrillic-LLama | [h-j-han/Llama2-7B-VocADT-50k-Cyrillic](https://huggingface.co/h-j-han/Llama2-7B-VocADT-50k-Cyrillic) | [Llama](https://huggingface.co/meta-llama/Llama-2-7b-hf) | 50k | Russian (ru), Bulgarian (bg), Ukrainian (uk), Kazakh (kk), English (en) |
-
-
-## Quick Start
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
-# model_name = "meta-llama/Llama-2-7b-hf" # Base Model
-model_name = "h-j-han/Llama2-7B-VocADT-50k-Cyrillic" # Vocabulary Adapted Model
-tokenizer = AutoTokenizer.from_pretrained(model_name)
-model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
-
-prefix = "\nEnglish: Hello!\nUkrainian: Добрий день!\nEnglish: How are you?\nUkrainian: Як справи?\nEnglish: "
-line = "Do you speak English?"
-suffix = f"\nUkrainian:"
-prompt = prefix + line + suffix
-
-inputs = tokenizer(prompt, return_tensors="pt")
-for item in inputs:
-    inputs[item] = inputs[item].cuda()
-outputs = model.generate(**inputs, max_new_tokens=6)
-print(tokenizer.decode(outputs[0], skip_special_tokens=True))
-```
-
-## Reference
-We provide code in Github repo: https://github.com/h-j-han/VocADT  
-Also, please find details in this paper:
-```
-@misc{han2024vocadt,
-      title={Adapters for Altering LLM Vocabularies: What Languages Benefit the Most?}, 
-      author={HyoJung Han and Akiko Eriguchi and Haoran Xu and Hieu Hoang and Marine Carpuat and Huda Khayrallah},
-      year={2024},
-      eprint={2410.09644},
-      archivePrefix={arXiv},
-      primaryClass={cs.CL},
-      url={https://arxiv.org/abs/2410.09644}, 
-}
+---
+license: mit
+---
+---
+license: mit
+datasets:
+- allenai/MADLAD-400
+language:
+- en
+- ru
+- bg
+- uk
+- kk
+base_model:
+- meta-llama/Llama-2-7b-hf
+---
+VocADT is a solution for vocabulary adaptation using adapter modules that are trained to learn the optimal linear combination of existing embeddings while keeping the model’s weights fixed.
+VocADT offers a flexible and scalable solution without requiring external resources or language constraints.
+
+
+## New Vocabulary Adapted Models
+Only the input/output embeddings are replaced, while all other original weights of base model remain fixed.
+These are the merged version: after training the adapters, we merge the original embeddings with the adapter to generate the new embeddings.
+| Name | Adapted Model | Base Model | New Vocab Size | Focused Languages |
+|---|---|---|---|---|
+| VocADT-Latin-Mistral | [h-j-han/Mistral-7B-VocADT-50k-Latin](https://huggingface.co/h-j-han/Mistral-7B-VocADT-50k-Latin) | [Mistral](https://huggingface.co/mistralai/Mistral-7B-v0.1) | 50k | Swahili (sw), Indonesian (id), Estonian (et), Haitian Creole (ht), English (en)|
+| VocADT-Mixed-Mistral | [h-j-han/Mistral-7B-VocADT-50k-Mixed](https://huggingface.co/h-j-han/Mistral-7B-VocADT-50k-Mixed) | [Mistral](https://huggingface.co/mistralai/Mistral-7B-v0.1) | 50k | Korean (ko), Greek (el), Russian (ru), Bulgarian (bg), English (en) |
+| VocADT-Cyrillic-Mistral | [h-j-han/Mistral-7B-VocADT-50k-Cyrillic](https://huggingface.co/h-j-han/Mistral-7B-VocADT-50k-Cyrillic) | [Mistral](https://huggingface.co/mistralai/Mistral-7B-v0.1) | 50k | Russian (ru), Bulgarian (bg), Ukrainian (uk), Kazakh (kk), English (en) |
+| VocADT-All-Mistral | [h-j-han/Mistral-7B-VocADT-50k-All](https://huggingface.co/h-j-han/Mistral-7B-VocADT-50k-All) | [Mistral](https://huggingface.co/mistralai/Mistral-7B-v0.1) | 50k | Swahili (sw), Indonesian (id), Estonian (et), Haitian Creole (ht), Korean (ko), Greek (el), Russian (ru), Bulgarian (bg), Ukrainian (uk), Kazakh (kk), English (en) |
+|||||
+| VocADT-Latin-LLama | [h-j-han/Llama2-7B-VocADT-50k-Latin](https://huggingface.co/h-j-han/Llama2-7B-VocADT-50k-Latin) | [Llama](https://huggingface.co/meta-llama/Llama-2-7b-hf) | 50k | Swahili (sw), Indonesian (id), Estonian (et), Haitian Creole (ht), English (en)|
+| VocADT-Mixed-LLama | [h-j-han/Llama2-7B-VocADT-50k-Mixed](https://huggingface.co/h-j-han/Llama2-7B-VocADT-50k-Mixed) | [Llama](https://huggingface.co/meta-llama/Llama-2-7b-hf) | 50k | Korean (ko), Greek (el), Russian (ru), Bulgarian (bg), English (en) |
+| VocADT-Cyrillic-LLama | [h-j-han/Llama2-7B-VocADT-50k-Cyrillic](https://huggingface.co/h-j-han/Llama2-7B-VocADT-50k-Cyrillic) | [Llama](https://huggingface.co/meta-llama/Llama-2-7b-hf) | 50k | Russian (ru), Bulgarian (bg), Ukrainian (uk), Kazakh (kk), English (en) |
+
+
+## Quick Start
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# model_name = "meta-llama/Llama-2-7b-hf" # Base Model
+model_name = "h-j-han/Llama2-7B-VocADT-50k-Cyrillic" # Vocabulary Adapted Model
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
+
+prefix = "\nEnglish: Hello!\nUkrainian: Добрий день!\nEnglish: How are you?\nUkrainian: Як справи?\nEnglish: "
+line = "Do you speak English?"
+suffix = f"\nUkrainian:"
+prompt = prefix + line + suffix
+
+inputs = tokenizer(prompt, return_tensors="pt")
+for item in inputs:
+    inputs[item] = inputs[item].cuda()
+outputs = model.generate(**inputs, max_new_tokens=6)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Reference
+We provide code in Github repo: https://github.com/h-j-han/VocADT  
+Also, please find details in this paper:
+```
+@misc{han2024vocadt,
+      title={Adapters for Altering LLM Vocabularies: What Languages Benefit the Most?}, 
+      author={HyoJung Han and Akiko Eriguchi and Haoran Xu and Hieu Hoang and Marine Carpuat and Huda Khayrallah},
+      year={2024},
+      eprint={2410.09644},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2410.09644}, 
+}
 ```