In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

Ex Input:
We use a standard set of supervised as well as unsupervised benchmark tasks from the literature to evaluate our trained models, following BIBREF16 Sentence embeddings are evaluated for various supervised classification tasks as follows. The predefined training split is used to tune the L2 penalty parameter using cross-validation and the accuracy and F1 scores are computed on the test set.  We perform unsupervised evaluation of the learnt sentence embeddings using the sentence cosine similarity, on the STS 2014 BIBREF31 and SICK 2014 BIBREF32 datasets. These similarity scores are compared to the gold-standard human judgements using Pearson's INLINEFORM0 BIBREF33 and Spearman's INLINEFORM1 BIBREF34 correlation scores.  
 Question: What metric is used to measure performance?

Ex Output:
Accuracy and F1 score for supervised tasks, Pearson's and Spearman's correlation for unsupervised tasks


Ex Input:
For automatically evaluating our methods, we propose to use widely used metric for image/video captioning. This is because the proposed CommonGen task can be regarded as also a caption task where the context are incomplete scenes with given concept-sets. Therefore, we choose BLEU-3/4 BIBREF28, ROUGE-2/L BIBREF29, CIDEr BIBREF30, and SPICE BIBREF31 as the main metrics. Apart from these classic metrics, we also include a novel embedding-based metric named BERTScore BIBREF32. To make the comparisons more clear, we show the delta of BERTScore results by subtracting the score of merely using input concept-sets as target sentences, named $\triangle $BERTS. 
 Question: What automatic metrics are used for this task?

Ex Output:
BLEU-3/4 ROUGE-2/L CIDEr SPICE BERTScore


Ex Input:
SIGHAN Bakeoff defines two types of evaluation settings, closed test limits all the data for learning should not be beyond the given training set, while open test does not take this limitation BIBREF21. 
 Question: What is meant by closed test setting?

Ex Output:

closed test limits all the data for learning should not be beyond the given training set, while open test does not take this limitation


