LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback

To democratize <PRE_TAG>large language models (LLMs)</POST_TAG> to most natural languages, it is imperative to make these models capable of understanding and generating texts in many languages, in particular low-resource ones. While recent multilingual LLMs demonstrate remarkable performance in such capabilities, these LLMs still support a limited number of human languages due to the lack of training data for <PRE_TAG>low-resource languages</POST_TAG>. Moreover, these LLMs are not yet aligned with human preference for downstream tasks, which is crucial for the success of LLMs in English. In this paper, we introduce xLLaMA-100 and xBLOOM-100 (collectively xLLMs-100), which scale the multilingual capabilities of LLaMA and BLOOM to 100 languages. To do so, we construct two datasets: a multilingual instruction dataset including 100 languages, which represents the largest language coverage to date, and a <PRE_TAG>cross-lingual human feedback</POST_TAG> dataset encompassing 30 languages. We perform <PRE_TAG><PRE_TAG>multilingual instruction tuning</POST_TAG></POST_TAG> on the constructed instruction data and further align the LLMs with human feedback using the <PRE_TAG>DPO algorithm</POST_TAG> on our <PRE_TAG>cross-lingual human feedback</POST_TAG> dataset. We evaluate the multilingual understanding and generating capabilities of xLLMs-100 on five multilingual benchmarks. Experimental results show that xLLMs-100 consistently outperforms its peers across the benchmarks by considerable margins, defining a new state-of-the-art multilingual LLM that supports 100 languages.