Solving Quantitative Reasoning Problems with Language Models

Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding. Nevertheless, state-of-the-art models have generally struggled with tasks that require <PRE_TAG><PRE_TAG><PRE_TAG>quantitative reasoning</POST_TAG></POST_TAG></POST_TAG>, such as solving mathematics, science, and engineering problems at the college level. To help close this gap, we introduce Minerva, a large language model pretrained on general natural language data and further trained on technical content. The model achieves state-of-the-art performance on technical benchmarks without the use of external tools. We also evaluate our model on over two hundred undergraduate-level problems in physics, <PRE_TAG><PRE_TAG>biology</POST_TAG></POST_TAG>, <PRE_TAG><PRE_TAG>chemistry</POST_TAG></POST_TAG>, <PRE_TAG><PRE_TAG>economics</POST_TAG></POST_TAG>, and other sciences that require <PRE_TAG><PRE_TAG><PRE_TAG>quantitative reasoning</POST_TAG></POST_TAG></POST_TAG>, and find that the model can correctly answer nearly a third of them.