Update README.md
Browse files
README.md
CHANGED
@@ -12,6 +12,10 @@ datasets:
|
|
12 |
|
13 |
[HuggingFace](https://huggingface.co/) is one of the most useful libraries for a NLP researcher / developer as it provides numerous pre-trained models, datasets, and tons of utility functions for NLP. In this repository, I'm trying to setup a complete pipeline for a Machine Learning project and the task I've chosen for the setup is Question Generation for Paragraphs. This is a seq2seq task for which I intend to fine-tune a pre-trained encoder-decoder Transformer model for Extractive Summarization like BART / Pegasus. More specifically, I'm finetuning the `sshleifer/distilbart-cnn-6-6` model on the SQuAD dataset.
|
14 |
|
|
|
|
|
|
|
|
|
15 |
# Dataset
|
16 |
|
17 |
The goal of Question Generation is to generate a valid and fluent question according to a given passage and the target answer. Hence, the input to the model will be a passage context and an answer, and the output / target will be the question for the given answer. Question Generation can be used in many scenarios, such as automatic tutoring systems, improving the performance of Question Answering models and enabling chat-bots to lead a conversation. The final dataset is created by taking the union of the following Question Answering Datasets. The dataset must have the following three columns: context, question, answer.
|
@@ -50,7 +54,3 @@ The Natural Questions corpus is a question answering dataset by Google. Each exa
|
|
50 |
## [TriviaQA](http://nlp.cs.washington.edu/triviaqa/) [Not Used]
|
51 |
|
52 |
TriviaQA is a realistic text-based question answering dataset which includes 950K question-answer pairs from 662K documents collected from Wikipedia and the web. This dataset is more challenging than standard QA benchmark datasets such as Stanford Question Answering Dataset (SQuAD), as the answers for a question may not be directly obtained by span prediction and the context is very long. TriviaQA dataset consists of both human-verified and machine-generated QA subsets.
|
53 |
-
|
54 |
-
# Usage
|
55 |
-
|
56 |
-
The input format is as follows: `[answer] <s> [passage]`. The model will predict the question that corresponds to the answer from the passage.
|
|
|
12 |
|
13 |
[HuggingFace](https://huggingface.co/) is one of the most useful libraries for a NLP researcher / developer as it provides numerous pre-trained models, datasets, and tons of utility functions for NLP. In this repository, I'm trying to setup a complete pipeline for a Machine Learning project and the task I've chosen for the setup is Question Generation for Paragraphs. This is a seq2seq task for which I intend to fine-tune a pre-trained encoder-decoder Transformer model for Extractive Summarization like BART / Pegasus. More specifically, I'm finetuning the `sshleifer/distilbart-cnn-6-6` model on the SQuAD dataset.
|
14 |
|
15 |
+
# Usage
|
16 |
+
|
17 |
+
The input format is as follows: `[answer] <s> [passage]`. The model will predict the question that corresponds to the answer from the passage.
|
18 |
+
|
19 |
# Dataset
|
20 |
|
21 |
The goal of Question Generation is to generate a valid and fluent question according to a given passage and the target answer. Hence, the input to the model will be a passage context and an answer, and the output / target will be the question for the given answer. Question Generation can be used in many scenarios, such as automatic tutoring systems, improving the performance of Question Answering models and enabling chat-bots to lead a conversation. The final dataset is created by taking the union of the following Question Answering Datasets. The dataset must have the following three columns: context, question, answer.
|
|
|
54 |
## [TriviaQA](http://nlp.cs.washington.edu/triviaqa/) [Not Used]
|
55 |
|
56 |
TriviaQA is a realistic text-based question answering dataset which includes 950K question-answer pairs from 662K documents collected from Wikipedia and the web. This dataset is more challenging than standard QA benchmark datasets such as Stanford Question Answering Dataset (SQuAD), as the answers for a question may not be directly obtained by span prediction and the context is very long. TriviaQA dataset consists of both human-verified and machine-generated QA subsets.
|
|
|
|
|
|
|
|