A prominent weakness of modern language models (LMs) is their tendency to generate factually incorrect text, which hinders their usability. A natural question is whether such factual errors can be detected automatically. Inspired by truth-seeking mechanisms in law, we propose a  factuality evaluation framework for LMs that is based on cross-examination.
Our key idea is that an incorrect claim is likely to result in inconsistency with other claims that the model generates. To discover such inconsistencies, we facilitate a multi-turn interaction between the LM that generated the claim and another LM (acting as an examiner) which introduces questions to discover inconsistencies.
We empirically evaluate our method on factual claims made by multiple recent LMs on four benchmarks, finding that it
outperforms existing methods and baselines, often by a large gap.
Our results demonstrate the potential of using interacting LMs to capture factual errors.

§ INTRODUCTION

An example of our  approach. The first line shows the statement made by the   LLM. Then an interaction between the  and  takes place, and the  arrives at a conclusion whether the original statement was correct or not (here it concludes that it was a false statement).

Modern language models (LMs) often generate inconsistent <cit.>, non-attributable <cit.>, or factually incorrect text <cit.>, thus negatively impacting the reliability of these models <cit.>.
This has prompted the community to develop methods that calibrate the confidence of model predictions to better align with their quality <cit.>. For example, prior methods have used probabilistic approaches <cit.> clustering <cit.>, fine-tuning <cit.> and in-context learning <cit.>.

In this work, we take a different approach to this problem, motivated by truth-seeking mechanisms in law. Specifically, we consider the setting where a witness is cross-examined in order to check whether their statement is factually correct or not. In such a setting, the examiner asks questions that aim to lead towards contradictory statements by the witness, while a contradiction implies that the witness lied at least in some of the statements, hence the well known quote “Were you lying then or are you lying now?” <cit.>.

To employ this mechanism to LM factual calibration, we propose the following setting, illustrated in Figure <ref>. Our goal is to check whether a statement made by an LM (The Greek god of marriage is Hera) is factually correct. We refer to the model that generated this statement as the . To check whether this fact is correct, we use another LM, called , to conduct a cross-examination of . Concretely, we craft designated prompts to facilitate a multi-turn interaction between the two LMs, where  issues questions (e.g., Is Hera associated with marriage in any way?) to  to check the veracity of the original statement. The examination is concluded by a decision from  as to whether the original claim was correct or not.[In practice,  and  can be the same LM (e.g., ) that is prompted in two different ways to define its different roles.]

Our problem setting is related to that of calibration <cit.>, where the goal is to predict the probability at which a model will err.
However, unlike previous approaches to this problem, we use  text generated by LMs.
Our approach is motivated by the intuition that calibration is actually an elaborate reasoning process where one checks the level of support that a fact has based on other statements the model believes. We argue that such complex reasoning is naturally performed via the strong conversational skills of modern LMs.

We use our method to detect errors in LM generation in the context of factual question-answering. Our experiments with several recent LMs – ,  <cit.>, and  <cit.> – show that cross-examination effectively detects factually incorrect claims generated by LMs. Specifically, across multiple datasets and examination settings, it detects over 70% of the incorrect claims while maintaining a high precision of $>$80%, outperforming strong baselines by a large gap.

Further analysis shows that examiner LMs introduce multiple questions throughout the examination, and employ various strategies to reveal inconsistencies, including question paraphrasing, validation of implicated arguments, claim decomposition, and requests for evidence.

To conclude, our contributions are (a) framing the task of factuality testing as an interaction between two LMs, (b) proposing a concrete implementation of this interaction via the use of one LM with different prompts in a zero-shot setting, and (c) demonstrating improved factuality detection accuracy across several benchmarks.

§ LM CROSS-EXAMINATION

Our goal is to employ an “examiner” LM () to evaluate claims generated by another LM (). To this end, we leverage the recent success of prompting <cit.>, to facilitate a cross-examination setting between the two LMs.
In such a setting,  should introduce questions with the objective of revealing inconsistencies with respect to an initial claim made by . Such inconsistencies can be considered as a signal for uncertainty of  in its original claim, and thus, can be used to assess whether its original statement was correct.

Given an  LM and a claim $C$ generated by an , our method establishes a multi-turn interaction between the LMs, where at each turn the other LM is prompted with a designated prompt that incorporates the outputs from previous turns. This interaction continues until the examiner has no further questions and can provide its final decision.
To establish a meaningful interaction that reveals possible inconsistencies, we define three stages for the examination, each guided by a specific prompt. As part of each prompt for  or , we provide the outputs generated in the previous rounds for context. We next describe the examination stages in detail, with the overall process illustrated in Figure <ref>.

Stage 1: Setup
The examination begins by “assigning” the  its role. Namely, describing the task setting, providing it with the 's claim, and asking it to generate questions for the .[We observe that this effectively steers  to ask natural questions directly related to the given claim $C$ (<ref>).]

Next, we feed the questions generated by , one at a time, to , concatenated to the following instructions: . The response from  yields a set of answers to the questions from .

The three-stage  process of cross-examination between the  and , where the factuality of a claim $C$ generated by  is estimated by .

Stage 2: Follow-up Questions
We next feed  with the answers generated by  to its initial questions, and ask  whether it has any follow-up questions.
Notably, outputs from  at this stage are conditioned on the previous output from .
If the answer from  is “Yes”, we then further prompt it to obtain more questions.
This phase is conducted iteratively, until either  declares it has no follow-up questions, or the number of turns has reached a threshold.[We use a maximum of five turns in our experiments.]

Stage 3: Factuality Decision
Once no further questions are obtained from , we prompt it to conclude whether the claim $C$ is true or false.
Specifically, we request it to reply with either “correct” or “incorrect” as its final conclusion.
In cases where the examiner does not output either of “correct” or “incorrect”, we consider its final decision to be a rejection of the claim.
Typically though, we observe that the examiner follows the instructions and indeed generates a definitive conclusion (see statistics in <ref>).

§ RELATED WORK

Attribution and Fact Checking
Our goal is closely related to works on attribution and fact verification. Namely, checking if a LM-generated text is faithful to some source text <cit.>. This problem has been addressed via several approaches, including question generation <cit.>, NLI <cit.>, data augmentation <cit.>, and planning schemes that allow the model to self-edit its own generation <cit.>.
Unlike these works, we are not assuming any reference text or an external knowledge base.
Instead, we directly check if the LM's claim is likely to be correct, by probing
the model for inconsistencies. Our approach also uses multi-turn dialogue as a key component.

Model Calibration
A key challenge with prediction models is to provide a probability of the answer being incorrect, a problem known as model calibration <cit.>. The problem of factual-error detection can be viewed as a variation of calibration, where instead of a continuous probability, we provide a binary prediction for whether the model is correct or not. This is also related to the setting of selective prediction, where a model can choose to abstain from answering a query <cit.>.
Common approaches to calibration are to perform various transformations on model logits <cit.>, and measuring uncertainty <cit.>.
More recent works have studied the use of LMs for providing calibration, by training them on statements known to be factually correct or incorrect. This “supervised” approach has been explored via fine-tuning <cit.> and in-context learning <cit.>.

Our work focuses on zero-shot factual error detection

that involves just two categories: predicting whether a model's claim is correct or incorrect. We propose a novel approach to this problem, using multi-turn LLM interaction.
While we focus on a binary setting, one could envision an extension of our approach to continuous outputs (for example, to output a probabilistic estimation for the correctness of the claim).

Multi-Agent LMs
Using multiple LMs in an interactive manner is a relatively new idea with many potential applications. It has been shown that LMs can utilize additional LMs or tools to better solve downstream tasks <cit.>. Additionally, <cit.> showed that in a social setting, LMs demonstrate certain social skills that emerge from this interaction, and <cit.> proposes that a LM can use a different model to instruct it when to “reflect” on its recent action, while performing a planned sequence of actions aimed at solving a given query. Intuitively, this model detects signs of hallucination or inefficient planning within the LM's trajectory.

Consistency Across Generations
LMs have been shown to generate inconsistent outputs given different prompt paraphrases <cit.>. Prior work showed that prompts can be automatically optimized to produce factually correct claims more robustly <cit.>. <cit.> utilized multiple generated paraphrases to gauge consistency <cit.>, and other works <cit.> further proposed training objectives to improve model consistency. Another approach to handling multiple outputs is via variants of decoding strategies <cit.>, or model ensembles <cit.>.
In our work, we build on these, assuming inconsistencies are more likely to occur with incorrect claims, and let an examiner model search for these by introducing questions to the examinee.

Chain of Thought Reasoning
Recent work has shown that LMs can be prompted to elaborate on their reasoning process, to self-ask themselves follow-up questions, before reaching a final conclusion, and that this could be exploited to improve mathematical, multi-hop and common-sense reasoning skills <cit.>, along with planning and problem-solving abilities <cit.>.
Another interesting approach to complex reasoning in LMs is recent work on Maieutic prompting <cit.>, that answers a question by recursively generating a set of facts and reasoning over those.

Our approach may be viewed as constructing an elaborate chain-of-thought explanation for the examinee's claim. However, we do not train this explanation via in-context or fine-tuning, and rather rely on different prompts for its generation.

Stage     GPT3 Prompt(s)     ChatGPT Prompt(s)

(1) Setup

(2) Follow-Up Questions

(3) Factuality Decision

Prompts provided to  in each stage of the examination, with respect to a claim $C$ by .

§ EXPERIMENTS

In this section, we conduct experiments on multiple datasets and models to evaluate our approach, focusing on the task of factual question-answering.

§.§ Experimental Setup

Factual Question Answering

One key use-case of LMs is answering questions seeking factual knowledge. For example, How old was Barack Obama when he was first elected?. In such cases, it is crucial for the model to answer the question correctly, or to indicate that it does not know the answer.
We thus evaluate our approach on several Question Answering and Fact Completion datasets. These are typically provided as a set of $(Q ,A)$ pairs of a question $Q$ and its ground-truth answer $A$.
Having gold answers allows us to evaluate if a predicted answer is factually correct or not, which can be used to evaluate our  approach.

To apply cross-examination in this setting, we first convert the answer predicted by the model into a  claim that can be provided as input to the examination procedure. Formally, given a question $Q$, if $Q$ is phrased as a fill-in-the-blank question (e.g. Bailey Peninsula is located in ), then we feed it to the  model  to obtain a prediction that completes the sentence and forms a claim. In cases where $Q$ is phrased as a question (e.g. Where is Bailey Peninsula located?), we prompt the model to provide an answer in a claim format with:
This process results in a claim $C$ that states the model's “belief” about the answer to $Q$. We then evaluate the truthfulness of $C$ through cross-examination, and compare the examiner's decision of whether $C$ is correct or not to the ground-truth correctness.

LAMA     TriviaQA     NQ     PopQA

53.9    48.4    33.8    24.9

79.8    74.2    50.1    43.9

80.9    77.2    53.3    45.6

Portion of factually correct claims by every  LM on each dataset.

Factuality Evaluation Labels
To evaluate our method, it is necessary to have “gold decisions” to compare the examiner's decisions against. Such labels can be obtained from the ground-truth answers in the data, namely, the decision for a claim $C$ is correct if it matches an evaluation of $C$ against the gold answer $A$.
To evaluate if the claim $C$ obtained for a question $Q$ is correct with respect to the ground-truth answer $A$, we first check if $A$ or any of its aliases (if provided as part of the dataset, e.g., “FC Tottenham” and “Tottenham Hotspur”) appears as a sub-string in $C$ <cit.>. Next, to avoid incorrect labels resulting from this automatic evaluation <cit.>, we manually review all the claims marked as incorrect in the first step, and fix any labeling mistakes. We also filter out any ambiguous or unclear claims generated by .

Examiner Evaluation
We evaluate how well the examiner detects claims that are factually incorrect, using the following metrics:[We say that the examiner “rejects” a claim if the examiner concludes that the claim is incorrect.]

* Precision: the portion of incorrect claims, out of the claims rejected by the examiner.

* Recall:
the portion of incorrect claims rejected by the examiner, out of all the incorrect claims.

* F1: the harmonic mean of precision and recall.

For completeness, we additionally report (in <ref>) the complementary Precision, Recall, and F1 scores with respect to detection of correct claims.

We consider the following datasets: LAMA <cit.>, TriviaQA <cit.>, Natural Questions (NQ) <cit.> and PopQA <cit.>. These datasets cover a wide range of queries, from real user queries (NQ), to trivia questions (TriviaQA), and subject-relation-object facts phrased as queries (LAMA, PopQA). We consider the closed-book open-ended setting, where we do not provide any context or answer choices to the model.
We evaluate our approach on 1,000 random examples from the test set (or from the development set if a test set is not available).[We use only a subset of examples due to the high cost of executing large LMs like , which we include in our experiments.]

In addition, we created a dataset of false claims to further test our approach. This “Falsehoods” dataset contains only wrong claims, created separately for each model ( and ) and for each of the four QA datasets. Concretely, given a model and a question $Q$, we prompt the model to generate a false answer (see <ref> for details). We verify that these are indeed incorrect claims by checking that the gold answer (and any of its aliases, if they exist) does not occur in the generated text. This yields a subset of examples that are realistic, namely, the answer matches the target type (e.g., “a city”) but is incorrect (see examples in Table <ref>). The examiner's decision for these examples should always be to reject.

False claim     Correct claim
[t]2mm8*[origin=c]90     Wanlockhead is the highest village in France because it is located in the French Alps.     Wanlockhead is the highest village in Scotland, a country in Europe.
Louis Oosthuizen the 2010 Open Golf Champion is American, because he was born in the United States.     Louis Oosthuizen, the 2010 Open Golf Champion, is South African.

[t]2mm6*[origin=c]90     The screenwriter for "Smile" was definitely Steven Spielberg.     The screenwriter for "Smile" was Jerry Belson.
Fontenay is located in the beautiful country of Antarctica.     Fontenay is located in France.

Example false claims generated by  for PopQA and by  for TriviaQA.

3cLAMA     3cTriviaQA      3cNQ     3cPopQA
1c     P      R     F1      P     R     F1      P      R     F1     P      R     F1

AYS                     $82.3$       $25.2$      $38.6$
$79.9$       $17.9$      $29.2$
$85.2$       $29.1$      $43.3$
$78.4$       $35.7$      $63.9$

IDK                     $49.1$       $52.4$      $50.7$
$48.7$       $66.5$      $56.2$
$62.5$       $60.7$      $61.6$
$70.0$       $61.1$      $65.2    $
$85.1$       $70.7$      $76.7$
$82.8$       $71.6$      $76.8$
$74.5$       $74.9$      $77.7$
$83.6$       $77.1$      $80.2$

(Majority)       $\mathbf{86.6}$       $\mathbf{75.8}$      $\mathbf{80.8}$
$\mathbf{84.5}$       $\mathbf{80.8}$      $\mathbf{82.6}$
$\mathbf{82.3}$       $\mathbf{76.1}$      $\mathbf{79.1}$
$\mathbf{87.0}$       $\mathbf{84.0}$      $\mathbf{85.4}$

- Follow-up      $83.8$       $68.1$      $75.1$
$82.3$       $69.7$      $75.5$
$74.8$       $72.1$      $73.4$
$82.0$       $73.3$      $77.4$

Precision (P), Recall (R), and F1 scores for  with  as  and , compared to baselines. The last row shows an ablation of our method without the follow-up questions stage.

3cLAMA     3cTriviaQA      3cNQ     3cPopQA
1c     P      R     F1      P     R     F1      P      R     F1     P      R     F1

AYS                     $74.8$       $17.9$      $28.9$
$80.3$       $19.8$      $31.8$
$74.9$       $20.7$      $32.3$
$74.6$       $22.7$      $34.8$

IDK                     $43.0$       $42.1$      $42.5$
$47.9$       $45.7$      $46.7$
$60.9$       $45.3$      $52.0$
$52.1$       $37.6$      $43.7$

Confidence-Based        $38.6$       $\mathbf{85.8}$      $53.2$
$39.6$       $\mathbf{84.4}$      $53.9$
$56.2$       $72.7$      $63.4$
$60.8$       $69.7$      $64.9$

IC-IDK                 $71.5$       $46.3$      $56.2$
$70.6$       $49.7$      $60.1$
$70.0$       $57.6$      $63.2$
$76.9$       $37.7$      $50.6$
$78.8$       $69.9$      $74.1$
$81.6$       $64.6$      $72.1$
$70.5$       $66.6$      $68.5$
$75.5$       $69.1$      $72.2$

(Majority)       $\mathbf{80.7}$       $77.9$      $\mathbf{79.3}$
$\mathbf{83.1}$       $72.1$      $\mathbf{77.2}$
$\mathbf{79.3}$       $\mathbf{76.8}$      $\mathbf{78.0}$
$\mathbf{82.2}$       $\mathbf{71.4}$      $\mathbf{76.4}$

- Follow-up      $76.4$       $71.1$      $73.7$
$78.7$       $64.8$      $71.1$
$66.6$       $70.1$      $68.3$
$70.9$       $65.8$      $68.3$

Precision (P), Recall (R), and F1 scores for  with  as  and , compared to baselines. The last row shows an ablation of our method without the follow-up questions stage.

We use  (),  () <cit.>, and -7B <cit.>, in three  vs.  cross-examination settings:  vs. ,  vs. , and  vs. . Notably, using the same LM as  and  (except for their prompts, which are different), provides a cleaner setting where both LMs share the same knowledge. The prompts used for each LM at every stage of the examination are shown in Table <ref>.

3cLAMA     3cTriviaQA      3cNQ     3cPopQA
1c     P      R     F1      P     R     F1      P      R     F1     P      R     F1

AYS                     $61.4$       $38.0$      $46.9$
$60.0$       $35.7$      $44.8$
$71.1$       $15.0$      $24.8$
$74.8$       $14.2$      $23.9$

IC-IDK                 $56.6$       $49.0$      $52.5$
$58.9$       $52.5$      $55.5$
$66.2$       $53.4$      $59.1$
$66.8$       $50.1$      $57.3$

IDK                     $61.6$       $44.8$      $51.9$
$62.0$       $32.9$      $43.0$
$64.4$       $12.1$      $20.4$
$66.7$       $16.8$      $26.8$

Confidence-Based        $54.9$       $\mathbf{76.7}$      $64.0$
$56.9$       $\mathbf{85.8}$      $68.4$
$64.4$       $63.5$    $63.9$
$64.6$       $53.6$      $58.6$
$81.1$       $66.4$      $73.0$
$80.1$       $70.8$      $75.2$
$79.3$       $65.5$      $71.7$
$84.9$       $73.6$      $78.8$

(Majority)       $\mathbf{82.9}$       $73.9$      $\mathbf{78.1}$
$\mathbf{80.3}$       $76.8$     $\mathbf{78.5}$
$\mathbf{83.7}$       $\mathbf{74.2}$      $\mathbf{78.7}$
$\mathbf{88.3}$      $\mathbf{77.4}$     $\mathbf{82.5}$

- Follow-up      $79.7$       $65.7$      $72.0$
$80.0$       $69.8$      $74.6$
$79.4$       $63.7$      $70.7$
$83.3$       $71.8$      $77.1$

Precision (P), Recall (R), and F1 scores for  with  as  and  as , compared to baselines. The last row is an ablation of our method without the follow-up questions stage.

For each setting, we compare  with recent methods for uncertainty detection and variants of our approach:

* Confidence-Based:
The prediction head of LMs outputs a probability for the predicted token. It is a common practice to use this probability as a measure of confidence in the prediction <cit.>. In our case, the LM generates a multi-token claim, and we calculate the confidence for the claim as the product of probabilities for all predicted tokens of the answer only.
In order to output a binary decision (i.e., is the claim correct or not), we optimize a threshold over the train dataset to maximize F1. Note that our examination approach does not require tuning any threshold.

* Are you sure? (AYS): Recent work <cit.> has shown that LMs can be trained to estimate their certainty in generated facts. Here, we use a zero-shot version of this approach where we directly “ask” the model whether it is sure. Specifically, we add the following prompt right after the claim generation: . Then we take the output as the prediction whether the claim is correct or not.

* I don't know (IDK): Recently, <cit.> showed that LMs might have the capability to self-correct themselves, when instructed to do so. Here we instruct the model to output “I don't know” if it is uncertain, by concatenating the following sentence to the original query: .
If the model answers `I don't know' we label the corresponding claim as false, and otherwise true.

* In-context IDK (IC-IDK): We teach the model to output that it doesn't know the answer, via in-context demonstrations. We follow <cit.> and test each of the queries within an in-context setting. For each query, we first provide the model with $K$ demonstrations, as $D$ of them labeled as “Don't know” examples, while the rest $K-D$ are provided with their gold answer from the dataset. The “Don't know” examples are randomly selected out of a set of examples the model failed on, while evaluating it on an held-out set of examples from the dataset in a zero-shot setting. Intuitively, these examples' answers are likely to be unknown to the model, hence we labeled them with “Don't know”. The model predictions are either a target text or “Don't know”. Based on the output, we generate a factuality label as in the IDK baseline above. Notably, this baseline requires labeled data for the in-context demonstrations, which is not necessary for our approach.

* : A single execution of our method, where we accept or reject the claim according to the examiner's final decision.

*  (Majority):
For a given claim, we apply our method three times (with the same  and ), using sampling generation for follow-up questions generation. We reject the claim in case at least two of the examinations concluded it is false.

Since output probabilities are not provided as part of the 's API, we cannot provide results for the Confidence-Based baselines in this case. Moreover, we observe that  often fails to understand the task of IC-IDK.

LAMA     TriviaQA     NQ     PopQA

65.7     98.4     89.9     83.1

(Majority)       75.8      98.5      92.0      88.0
83.6      97.9      90.4      88.8

(Majority)       87.1      98.6      94.2      93.9

Accuracy of  and  as  on false claims generated for each dataset.

§.§ Results

Tables <ref>, <ref>, <ref> show the results for the settings  vs. ,  vs. , and  vs. , respectively.
Across all settings, our method outperforms the baselines, often by a large gap. For example, it obtains $85.4$ F1 compared to $\leq65.2$ by baselines for  on PopQA (Table <ref>), and $77.2$ F1 compared to $\leq60.1$ for  on TriviaQA (Table <ref>).
Notably, the most substantial gains are in terms of recall, showing the superiority of our method in detecting factually incorrect claims (when compared to the baselines which achieve reasonable precision too).
Interestingly, we observe that  generally outperforms .

Last, Table <ref> shows the accuracy of our method and baselines on our Falsehood dataset. For both  and ,  successfully rejects the vast majority of the false claims, obtaining 87%-98% accuracy with  and 75%-99% with  across all datasets.

§.§ Ablations

We perform an ablation, where we remove the follow-up iterations in the examination process to gauge their benefit. Results are reported for  in Table <ref> (last row), showing a large decrease in performance (e.g. $78 \rightarrow 68.3$ in F1 for NQ and $77.2 \rightarrow 71.1$ for TriviaQA). Notably, recall scores are decreased by 6%-10%. Overall, this shows the importance of the follow-up questions issued by the examiner to assess the examinee's claim.

§ ANALYSIS OF CROSS-EXAMINATIONS

We analyze cross-examinations by  and  to better understand the success and failure cases of our method. We find that examiner LMs typically ask multiple questions in the examination, and perhaps surprisingly, apply different strategies to reveal inconsistencies.

Examination Statistics
Table <ref> provides statistics on the cross-examinations performed by  and . Both models introduce multiple queries (6-7 on average) during an examination, with typically 1-2 steps of follow-up questions, which are important for the examiner's decision (<ref>).
We also observe a non-negligible number of claims (9%-15%) where the examiner LM does not arrive at a concrete final decision (i.e., it does not generate “correct” or “incorrect” as the final decision, we reject the claim in those cases). In our qualitative analysis, we identify reasons that could explain these cases.

Qualitative Analysis
We manually analyze a sample of 96 examinations – 48 by each LM, with 6 correct and 6 incorrect examinations for each model and each dataset.
We observe the following trends (examples are in Table <ref>):

* Rephrasing the claim: In about $60\%$ of the examinations, both LMs introduce questions which are paraphrases of the original question. This supports the assumption that the  seeks  inconsistencies by generating variants of the original claim.

* Rephrasing Questions: In about half of the examinations, both LMs introduce questions that are similar to previously asked questions or have a different phrasing. This is a desirable behavior as it can reveal inconsistencies if the examinee provides a different answer for the same question.

* Validation of logical implications: The  asks  regarding implied arguments that must be true whenever the original claim is correct. This can be observed in $70\%$ of the correct detections of , and $87.5\%$ out of the correct detections of .

* Logical questions: The  decomposes the claim into multiple sub-questions which together compose a trajectory to validating it. Such decompositions appear in about 75% of the cases for  but only 10% in . We observe these in $33\%$ of the correct detections of , and $70\%$ for .

* Request for attribution: The  ask the  about the existence of external evidence to support the claim. This happens in about 30% of the cases for both LMs.

* Wrong intermediate answers: The  responds with factually incorrect answers to one or more of the questions originated by the . We observe this occurs mostly in cases where the original claim is false (it happens in only in about $14\%$ of the cases where the  is correct). In both models, this can be observed in about half of the cases where the claim is false and  has also been detected by the . Furthermore, it occurs in about $80\%$ of the cases where the  has accepted a false claim, and in $45\%$ where the  has rejected a correct claim.

We note that in most cases where  fails,  has provided incorrect information to . This might indicate that in those cases  has encoded a large set of factually wrong facts that are mutually consistent, thus making it hard for the  to detect inconsistencies.
Finally, the fact that  more commonly validates the claim through logical questions might be a key factor in its superiority over  in our setting.

§ CONCLUSION

We introduce , a method for zero-shot detection of factuality errors, inspired by the cross-examination practice employed in a court of law. Our method uses prompting to facilitate a multi-turn interaction between an examiner LM and an examinee LM, to reveal inconsistencies that imply factually incorrect claims. We evaluate  in the context of factual question answering, showing it substantially improves detection of factual errors made by LMs.

Our method builds on a fundamental connection between self-consistency (i.e., consistency of an LM with itself) and factual consistency (i.e., consistency between factual claims generated by an LM and ground-truth facts). We consider the LM itself as the source of information, and we test whether a claim it has generated is faithful and consistent with several other beliefs it has.

Our work can be extended in several ways.
First,  provides interpretable information about related beliefs of the model, which could be analyzed to understand what makes the model commit certain mistakes.
Second, one may incorporate several LM instances into the factuality detection process, rather the having only a single . Finally, one can train the  to generate questions more effectively.

§ LIMITATIONS

We note three limitations of our method . First, unlike other methods, it requires multiple queries of the examinee and examiner LMs, which could be costly when using external APIs such as those used in this work. This could be a key consideration when scaling this approach to large numbers of claims.

Second, for our method to succeed, both LMs ( and ), but mostly , should be able to follow instructions and have the ability to reason over information in a relatively long context. This skill is currently mostly demonstrated by larger models (>10B parameters) and thus our method may not perform as well for smaller models.

Last, any logical flaws in the examiner's operation are likely to affect the overall examination, potentially leading to inaccurate decisions. However, our experiments show that, even if such flaws occur, our method is still useful on average as it substantially improves factuality detection.
Nonetheless, developing safety mechanisms that detect and mitigate logical flaws is an important research direction, that we leave for future work.

§ ACKNOWLEDGEMENTS

We thank Roee Aharoni and Avi Caciularu for valuable feedback and constructive suggestions. This work is supported in part by the Israeli Science Foundation.


# Batch Clustering for Multilingual News Streaming

Mathis Linger
<EMAIL_ADDRESS>Mhamed Hajaiej
<EMAIL_ADDRESS>
###### Abstract

Nowadays, digital news articles are widely available, published by various
editors and often written in different languages. This large volume of diverse
and unorganized information makes human reading very difficult or almost
impossible. This leads to a need for algorithms able to arrange high amount of
multilingual news into stories. To this purpose, we extend previous works on
Topic Detection and Tracking, and propose a new system inspired from newsLens.
We process articles per batch, looking for monolingual local topics which are
then linked across time and languages. Here, we introduce a novel ”replaying”
strategy to link monolingual local topics into stories. Besides, we propose
new fine tuned multilingual embedding using SBERT to create crosslingual
stories. Our system gives monolingual state-of-the-art results on dataset of
Spanish and German news and crosslingual state-of-the-art results on English,
Spanish and German news.

##  1 Introduction

The rise of the Internet and social media has increased the number of news
articles available. This flow of information is difficult to ingest but can
yet be very valuable for real-time economics decisions. For instance, within
the bank industry, being able to extract the main information from numerous
and diverse news articles can help to assess risk linked to companies. This
leads to a need for scalable systems, able to organize a large, multi-source
and multilingual flow of news articles. The Topic Detection and Tracking (TDT)
task refers to techniques that automatically search, organize and structure
news from a variety of broadcast news media. More precisely, its aim is to
arrange an unordered multilingual stream of news articles into major events
clusters called stories.

Recently, a two-steps streaming system called newsLens [LH17] has been
proposed to group articles into stories. However, this system does not support
multilingual articles. Other works such as the one introduced by Staykovski et
al. [SBCMN19], investigate methods to represent news articles and link them
along time. Based on these works, we propose an extention of the newsLens
algorithm. Experiments on standard benchmark dataset introduced by Miranda et
al. [MZCB18] for news stream clustering show significant improvement over the
state-of-the-art for the two main tasks: monolingual and multilingual news
clustering.

Our proposed approach improves topic tracking and handles multilingual data.
Specifically, our contribution is twofold:

  * $\bullet$

We extend newsLens [LH17] with a new topic matching procedure refered to as
”replaying” strategy to link monolingual topics through time.

  * $\bullet$

We show that multilingual documents are best represented through fine tuned
DistilBERT [SDCW19] multilingual model using SBERT [RG19] triplet network
structure.

##  2 Related Work

This paper directly follows a growing body of work on Topic Detection and
Tracking (TDT). Most of these works aim to solve the TDT task processing
articles according to well defined pipelines.

In a first approach, Laban and Hearst [LH17] propose newsLens, a two-steps
streaming system which first extracts keywords to create topics using Louvain
community detection algorithm [BGLL08], and then solidify these local topics
clusters into stories by comparing their keywords distribution. Emphasis is
put on scalability, as the volume of news articles processed in their proposed
experimental setup is about 4 millions. However, the algorithm does not handle
multilingual articles and uses simple TF-IDF based method to compare articles.
Moreover, its performance was not formally evaluated at the time.

Later, Miranda et al. [MZCB18] introduce a novel method to cluster an incoming
stream of multilingual documents into monolingual and crosslingual stories.
Documents are embedded in two latent spaces, a ”monolingual space” and a
”multilingual space”, which are used to cluster articles into topics. Then,
new incoming articles can contribute to stories if they are close enough to
topics centroids. Alongside their proposed system, they introduce a
multilingual dataset adapted from Rupnik et al. [RML+15] containing articles
in English, Spanish and German which have been manually annotated with
monolingual and crosslingual story cluster labels. To the best of our
knowlege, no other multilingual benchmark dataset has been proposed for the
TDT task.

Recently, Staykovski et al. [SBCMN19] use the English part of the corpus from
Miranda et al. [MZCB18] to assess the importance of article representations
for news clustering. Among other, they show that sparse vector representation
with TF-IDF weighting yields better results than doc2vec-based dense
representation [LM14].

##  3 Our system

We extend newsLens with a per-batch procedure, where documents published
within a close range of time are processed to form local monolingual topics.
Monolingual stories are then created by linking topics across time (i.e.
across batches, Figure 1a) using topic centroids method from Miranda et al.
[MZCB18]. Finally, multilingual stories are created by aggregating monolingual
stories from different languages whose representations in a multilingual
latent space are close enough (Figure 1b).

Figure 1: General description of our system. Colors represent languages of
articles/stories. $S_{i,t}$ represents story $i$ created at time $t$.
$MS_{j,t}$ represents multilingual story $j$ created at time $t$.

###  3.1 Monolingual stories

In order to create local topics, we process articles per batch of close range,
computing similarities between each pair of articles and making use of a
community detection algorithm. Then, we link the local topics along time
thanks to a ”replaying” strategy based on topics centroids similarities.

####  3.1.1 Article representation

Following Miranda et al. [MZCB18] and Staykovski et al. [SBCMN19] who
demonstrated the inefficiency of dense features to cluster documents of a same
language, articles are represented using sparse TF-IDF features. Monolingual
representations for each document consist of 9 TF-IDF weighted bag of words
sub-vectors, corresponding to the entities, lemmas and tokens contained in the
title, body and title+body of each document. Contrary to previous work, we do
not use any time feature, time being implicitely taken into account by the
per-batch procedure. Please note that in all proposed experiments, we use the
same entities, lemmas and tokens as already extracted by Miranda et al.
[MZCB18] to ensure fair comparison of our proposed system.

####  3.1.2 Topics detection

To group articles into local topics, we build a non-oriented graph, where
nodes represent articles and edges are weighted by the similarity between
articles. More precisely, weights associated to edges are a linear combination
of the cosine similarities between each of the 9 TF-IDF based representative
sub-vectors of articles. Formally, the similarity function between two
articles $i$ and $j$ is computed as :

$\sum^{K}_{k=0}\beta_{k}\times\theta(d_{i}^{k},d_{j}^{k})$ (1)

Where $K$ is the number of sub-vectors used to represent an article (i.e.
$K=9$), $\beta_{k}$ are learned weights associated to sub-vector $k$, $\theta$
is the cosine similarity function and $d_{i}^{k}$ is the sub-vector $k$ of
article $i$.

In order to learn the best $\beta$ weights to aggregate the cosine
similarities of articles’ representations for each language, we fit a logistic
regression using the training part of the dataset. More precisely, for
articles of a same language, we compute the cosine similarities between sub-
vectors of each pair of articles. We then assign positive labels for pairs
which are indeed of the same story and a negative ones for pairs of different
stories.

Besides, using (1), we are able to compute similarities between each pair of
articles. The resulting adjacency matrix can be viewed as a graph displaying
weak links for pairs of articles which are of different stories and strong
ones for pairs of a same story. We apply the Louvain community detection
algorithm [BGLL08] to this graph in order to extract well delimited
communities of articles, which will be refered to as topics.

####  3.1.3 Linking topics through time

Once a batch of documents has been clustered into topics, topics are linked
across time (i.e. across batches) to form monolingual stories. To this end, we
introduce a ”replaying” strategy (Figure 2), based on the similarity between
articles in the current batch and topic centroids from previous batches. More
precisely, when topics are created within a batch of articles at time $t-1$,
we compute their centers as the average of each articles representative sub-
vectors (Figure 2a). Then, for a new batch of articles at $t$, we compute
similarities between all new articles and all topics centers at $t-1$ using
the formula introduced in (1). When a topic at $t-1$ has a similarity with a
new article at $t$ greater than a threshold $T_{1}$ (in our implementation
$T_{1}=0.43$ for English, $0.61$ for German and $0.52$ for Spanish documents),
we replay all the articles constituting the topic at $t-1$, i.e. we add those
articles to the current batch so that they can be considered during the new
round of topic detection at time $t$ (Figure 2b). This approach allows for the
emmergence of different topic behaviors across time (Figure 2c): indeed, a
previously created topic can subsist (eventually aggregating new articles) or
not; it can also be split into several new topics. Finally, two or more topics
can be merged into one.

Figure 2: Linking topics along time using the ”replaying” strategy.

###  3.2 From monolingual to multilingual stories

Whenever we create new monolingual stories, we try to link them with current
and past stories in other languages. To do so, we compute a common
representation for stories in different languages, and associate them solving
an optimal assignment problem.

####  3.2.1 Story representation

To represent each article in a multilingual space, we use the SBERT [RG19]
triplet network structure. We use the training part of our dataset in order to
create labeled sentence triplets: The anchor and the positive example are
articles in different languages coming from the same story, while the negative
example is an article in different language and story than the anchor. We fine
tune the multilingual DistilBERT [SDCW19] model using the concatenation of
title and body articles for 15 epochs on 6,000 semi-hard triplets selected at
the beginning of each epoch. We use a batch size of 8, a gradient accumulation
of 2 steps, Adam optimizer with learning rate $2e-5$ and using the MEAN
pooling strategy. Then, in order to get the representation of a monolingual
story, we average these representations over all articles within the story.

$s_{i}=\frac{1}{|S_{i}|}\sum_{j\in S_{i}}e_{j}$ (2)

Where $S_{i}$ is story $i$, and $e_{j}$ is our fine tuned multilingual
DistilBERT embedding corresponding to article $j$.

####  3.2.2 Linking monolingual into multilingual stories

Based on the conclusions of Miranda et al. [MZCB18], we use English as a pivot
language in order to link monolingual into crosslingual stories. More
precisely, we compute cosine similarities between non English and English
story embeddings. Assuming that, at maximum one story of a given language can
contribute to a multilingual story, we have to solve an optimal assignment
problem. Between two sets of stories in different languages, we have to find
the stories assignment between the two languages such that the sum of the
similarities (resp. distances) of the linked story pairs is the highest (resp.
the lowest) possible. We use the Hungarian algorithm [Kuh55] to solve this
problem in polynomial time. More precisely, we define the cost function as the
distance matrix (1 - similarity matrix) between stories of two different
languages. Since some stories may not be related between two languages, we
allow assignments only if the distance between two stories is less than a
threshold $T_{2}$ ( $T_{2}$ is set to 0.22 in our implementation). We make
this connection between monolingual stories each time we receive a new batch
of articles, taking into account all monolingual stories not already assigned
to multilingual stories and not older than 4 batches.

##  4 Experimental setup

###  4.1 Dataset

We assess the effectiveness of our proposed approach on the standard
multilingual dataset introduced by Miranda et al. [MZCB18]. It is a collection
of 33,807 news articles in three languages: English, Spanish and German. These
articles are labeled by language and by story. Stories are multilingual, i.e.
that they may contain articles from several languages. The training set
contains 20,803 articles and the test set 13,004 articles. We further divide
the training set in two: a train part to learn the $\beta$ weights of the
linear combinations to aggregate similarities between articles (1) and a
development part to set the hyper parameter $T_{1}$ and the resolution
parameter of the Louvain algorithm (Section 3.1) as well as threshold $T_{2}$
(Section 3.2). In order to set these parameters, we perform a grid search
maximizing the average between standard and BCubed F1 scores. Table 1 presents
descriptive statistics of the dataset.

Table 1: Statistics for the train, development and test datasets.
Partition |  Language |  |  Nb of
---
documents
|  Avg nb of
---
words (std)
|  Nb of
---
of clusters
|  Avg cluster
---
size (std)
Train | English | 5,804 | 453 (359) | 296 | 20 (25)
Spanish | 2,278 | 357 (219) | 208 | 11 (7)
German | 1,879 | 286 (212) | 188 | 10 (6)
All | 9,961 | 399 (315) | 557 | 18 (20)
Dev | English | 6,429 | 438 (397) | 297 | 22 (37)
Spanish | 2,249 | 375 (264) | 208 | 11 (11)
German | 2,164 | 295 (227) | 189 | 11 (7)
All | 10,842 | 397 (348) | 557 | 19 (36)
Test | English | 8,726 | 546 (518) | 222 | 39 (88)
Spanish | 2,177 | 412 (358) | 149 | 15 (21)
German | 2,101 | 458 (496) | 118 | 18 (45)
All | 13,004 | 509 (494) | 381 | 34 (99)

###  4.2 Results

In order to assess the performances of our system, we report the standard as
well as the BCubed111Unlike the classic version, the BCubed version of
precision, recall and F1 score favors solutions that (i) make errors in
clusters with already many errors (ii) make errors in a large clusters rather
than in small ones. [AGAV08] precision, recall and F1 score. We evaluate our
system for two tasks: monolingual and multilingual news clustering.

####  4.2.1 Monolingual results

First, we can observe that for each language, our method produces a number of
clusters closer to the reality. Then, on English documents, our method
generates the best F1 and accuracy scores. However, when looking at BCubed
metrics, we can see that our system ranks second after the method introduced
in Staykovski et al. [SBCMN19]. Nevertheless, for the two other languages
which are German and Spanish, our method surpasses the system introduced in
Miranda et al. [MZCB18]. Indeed, it displays better F1 and BCubed F1 scores,
with improvements of 1.51 points for the F1 and 1.08 points for the BCubed F1
scores on German articles. Sometimes, even if our system is less precise than
the one of Miranda et al. [MZCB18], it displays a much higher recall, yielding
better standard and BCubed F1 scores. Notice that only 2% of our articles are
replayed using our ”replaying” strategy. This insure that we are not
performing a clustering from scratch each time we receive new articles.

Table 2: Monolingual clustering results on the test dataset. F1 is F1 score, P
stands for precision and R for recall. Best results are in bold.
| Systems | Bcubed | Standard | Nb of
---|---|---|---|---
|  | F1 | P | R | F1 | P | R | Clusters
English | newsLens |  89.76 |  94.37 |  85.58 |  95.09 |  95.90 |  94.30 |  873
Miranda et al. | 92.36 | 94.57 | 90.25 | 94.03 | 98.14 | 90.25 | 326
Staykovski et al. | 94.41 | 95.16 | 93.66 | 98.11 | 97.60 | 98.63 | 484
Ours | 93.86 | 94.19 | 93.55 | 98.31 | 98.21 | 98.42 | 298
Spanish | newsLens | - | - | - | - | - | - | -
Miranda et al. | 91.61 | 96.44 | 87.25 | 96.83 | 97.01 | 96.65 | 281
Staykovski et al. | - | - | - | - | - | - | -
Ours | 91.79 | 93.56 | 90.08 | 97.68 | 98.02 | 97.34 | 267
German | newsLens | - | - | - | - | - | - | -
Miranda et al. | 93.64 | 98.92 | 88.90 | 97.19 | 99.86 | 94.67 | 229
Staykovski et al. | - | - | - | - | - | - | -
Ours | 94.72 | 95.13 | 94.31 | 98.70 | 99.16 | 98.24 | 205

####  4.2.2 Crosslingual results

In the multilingual clustering setting, we compare our system to the one of
Miranda et al. [MZCB18], which is the only system handling multilingual news
articles. We can see that our system displays better F1 (+2.5 points),
precision and recall scores. This result shows that our improved system is
able to both better organize monolingual stories, and link these stories over
languages making use of our fine tuned DistilBERT embedding model.

Table 3: Crosslingual clustering results on the test dataset. Best results are
in bold.
Systems | BCubed | Standard | Nb of
---|---|---|---
| F1 | P | R | F1 | P | R | Clusters
Miranda et al. | - | - | - | 84.0 | 83.0 | 85.0 | -
Ours |  82.06 |  80.25 |  83.97 |  86.49 |  85.11 |  87.92 | 606

##  5 Conclusion and Future Work

We described a new method to cluster multilingual news articles into stories.
We processe articles per batch as in newsLens, and naturally link found topics
along time by maintaining centroids for monolingual clusters. More precisely,
we introduced a new ”replaying” strategy to link monolingual topics into
stories, and then create crosslingual stories by embedding articles thanks to
SBERT [RG19]. Our system gives both monolingual and crosslingual state-of-the-
art results on the English, Spanish and German dataset introduced by Miranda
et al. [MZCB18].

In future work, we plan to challenge the TF-IDF based representation of
monolingual articles using fine tuned SBERT [RG19] embeddings. Moreover, it
would be interesting to assess computational efficiency of different systems
by testing them on bigger news dataset.

## Acknowledgements

The authors would also like to thank Mr. Clément Rebuffel, Mr. Pirashanth
Ratnamogan, and Mr. Bruce Delattre from BNP Paribas for their valuable
comments and suggestions.

## References

  * [AGAV08] Enrique Amigó, Julio Gonzalo, Javier Artiles, and M. Felisa Verdejo.  A comparison of extrinsic clustering evaluation metrics based on formal constraints.  volume 12, pages 461–486, 2008.
  * [BGLL08] Vincent D. Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre.  Fast unfolding of communities in large networks.  2008\.
  * [Kuh55] Harold W. Kuhn.  The Hungarian Method for the Assignment Problem.  volume 2, pages 83–97, March 1955.
  * [LH17] Philippe Laban and Marti A. Hearst.  newslens: building and visualizing long-ranging news stories.  In NEWS@ACL, 2017.
  * [LM14] Quoc V. Le and Tomas Mikolov.  Distributed representations of sentences and documents.  volume 1405.4053, 2014.
  * [MZCB18] Sebastião Miranda, Arturs Znotins, Shay B. Cohen, and Guntis Barzdins.  Multilingual clustering of streaming news.  In EMNLP, 2018.
  * [RG19] Nils Reimers and Iryna Gurevych.  Sentence-bert: Sentence embeddings using siamese bert-networks.  In EMNLP/IJCNLP, 2019.
  * [RML+15] Jan Rupnik, Andrej Muhic, Gregor Leban, Blaz Fortuna, and Marko Grobelnik.  News across languages - cross-lingual document similarity and event tracking.  In IJCAI, 2015.
  * [SBCMN19] Todor Staykovski, Alberto Barrón-Cedeño, Giovanni Da San Martino, and Preslav Nakov.  Dense vs. sparse representations for news stream clustering.  In Text2Story@ECIR, 2019.
  * [SDCW19] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf.  Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.  ArXiv, abs/1910.01108, 2019.