TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question based on the context. You have to classify the questions into "Extractive", "Abstractive", or "Yes-no" questions. Extractive questions can be answered by concatenating extracts taken from a context into a summary while answering abstractive questions involves paraphrasing the context using novel sentences. Yes-no question is a question whose expected answer is one of two choices, one that affirms the question and one that denies the question. Typically, the choices are either yes or no.
PROBLEM: We introduce a surrogate training objective that avoids these problems and as a result is fully continuous.  Specifically, we form a continuous function softLB that seeks to approximate the result of running our decoder on input INLINEFORM0 and then evaluating the result against INLINEFORM1 using INLINEFORM2 . 
 Question: Do they provide a framework for building a sub-differentiable for any final loss metric?

SOLUTION: Yes-no

PROBLEM: We employ precision, recall, F1 and accuracy for evaluation metrics. 
 Question: what evaluation metrics are reported?

SOLUTION: Extractive

PROBLEM: We showed in BIBREF22 that Pearson correlation is badly suited for STS. Instead, we compute the Spearman's rank correlation between the cosine-similarity of the sentence embeddings and the gold labels. 
 Question: What metrics are used for the STS tasks?

SOLUTION:
Extractive

