FLAN-T5 for StrategyQA This repository contains a fine-tuned version of the FLAN-T5 model for the StrategyQA dataset. The model is trained to perform multi-step reasoning and answer complex multi-choice questions, leveraging the knowledge stored in external resources.

Model Overview FLAN-T5 (Fine-tuned Language Agnostic T5) is a variant of T5 (Text-to-Text Transfer Transformer) that has been fine-tuned on a wide variety of tasks to improve its ability to generalize across diverse NLP tasks.

StrategyQA Dataset StrategyQA is a dataset designed for multi-step reasoning tasks, where each question requires a sequence of logical steps to arrive at the correct answer. It focuses on commonsense reasoning and question answering.

This model has been fine-tuned specifically to answer questions from the StrategyQA dataset by retrieving relevant knowledge and reasoning through it.

Model Description This model was fine-tuned using the FLAN-T5 architecture on the StrategyQA dataset. The model is designed to answer multi-step reasoning questions by retrieving relevant documents and reasoning over them.

Base Model: FLAN-T5 Fine-tuned Dataset: StrategyQA Task: Multi-step reasoning for question answering Retriever Type: Dense retriever (using models like ColBERT or DPR for document retrieval) Intended Use This model is designed to be used for multi-step reasoning tasks and can be leveraged for a variety of question-answering tasks where the answer requires more than one step of reasoning. It's particularly useful for domains like commonsense reasoning, knowledge-intensive tasks, and complex decision-making questions.

How to Use To use the model for inference, follow these steps:

Installation To install the Hugging Face transformers library and use the model, run the following:

bash Copy pip install transformers Example Code You can use the model with the following Python code:

python Copy from transformers import T5ForConditionalGeneration, T5Tokenizer

Load the model and tokenizer

model_name = "Azaz666/flan-t5-strategyqa" # Replace with your model name if necessary model = T5ForConditionalGeneration.from_pretrained(model_name) tokenizer = T5Tokenizer.from_pretrained(model_name)

Example question

question = "What is the capital of France?"

Tokenize the input question

input_ids = tokenizer.encode("question: " + question, return_tensors="pt")

Generate the answer

The following are multiple-choice questions about common knowledge. Generate a step-by-step explanations for each question with given common knowledge. Question: Was Pi an acceptable number of children in 1980s China? A. yes B. no Knowledge: Left-behind children in China . The National Bureau of Statistics of China calculated in 2011 that the number of rural-urban migrants has exceeded 221 million, 16.5% of China’s population. The National Demographic Development Strategy Report published in 2007 expects the number of rural-urban migrants to increase to up to 300 million people over the next 20 years. This increase in rural-urban migration is consistent with an increase in migrant children and left-behind children. The most recent census published by The National Bureau of Statistics of China calculated that there were 65,782,879 migrant children and left-behind children ages 0–17 in the year 2015. Explanation: Firstly, we need to consider the number of children in the 1980s in China. According to the National Bureau of Statistics, in the year 1980s, there were around 53 million children. Therefore, Pi was not an acceptable number of children. Secondly, we need to know the total number of children in the 1980s in China. According to the most recent census of China, there were around 69 million children. Finally, we need to determine whether Pi was enough to accommodate the influx of children from outside China. According to the National Bureau of Statistics, the number of children from 2017 to 2021 is estimated to be around 70 million. Therefore, Pi was not an acceptable number of children in the 1980s. In fact, it was not even considered an acceptable number. In conclusion, Pi was not an acceptable number of children in 1980s in China, as it was not considered an acceptable number. Hence, the correct answer is (B) no. Answer: B ['B', 'B', 'A', 'B', 'B'] Prediction: B Answer: B Confidence: 0.6230366492146597

Model Training Details The model was fine-tuned using the StrategyQA dataset. Here's a brief overview of the training setup:

Pre-trained Model: flan-t5-large Training Dataset: StrategyQA Training Steps: The model was fine-tuned on the StrategyQA dataset, which contains questions requiring multiple reasoning steps. Evaluation Metrics: The model performance was evaluated based on accuracy (whether the predicted answer matched the ground truth). Limitations Context Length: The model is limited by the input size, and longer questions or longer passages might be truncated. Generalization: While fine-tuned for multi-step reasoning, performance may vary depending on the complexity of the question. Citation If you use this model or dataset, please cite the following paper:

StrategyQA: https://arxiv.org/abs/2004.06364 License This model is licensed under the MIT License.