52.22\
Syn-QG (w/o BT) & 26.8 & 43.93 & 53.4\
Syn-QG & 39.34 & 64.08 & 76.24\
SQUAD & 42.86 & 74.2 & 86.35\
Syn-QG (BT vs w/o-BT) & 28.78 & 55.18 & 67.81\

Most of the prior QG studies have evaluated the performance of the generated questions using automatic evaluation metrics used in the machine translation literature. We use the traditional BLEU scores [@papineni2002bleu] and compare the performance of Syn-QG on the SQuAD [@rajpurkar2016squad] test split created by @zhou2017neural. BLEU measures the average n-gram precision on a set of reference sentences. A question lexically and syntactically similar to a human question would score high on such n-gram metrics. Despite not utilizing any training data, Syn-QG performs better than the previous SOTA on two evaluation metrics BLEU-3 and BLEU-4 and close to SOTA on BLEU-1 and BLEU-2 (Table \[results-table\]) at the time of submission. The high scores obtained without conducting any training arguably shed a little light on the predictable nature of the SQuAD dataset too.

Besides SRL, Dependency, and NER templates, Syn-QG’s questions also arise from VerbNet’s predicates and PropBank’s descriptions, which indeed by nature describe events not mentioned explicitly within the fact. Like in Figure \[fig:vn1\], the sentence 