Language is Scary when Over-Analyzed: Unpacking Implied Misogynistic Reasoning with Argumentation Theory-Driven Prompts
Arianna Muti Federico Ruggeri Khalid Al-Khatib Alberto Barrón-Cedeño Tommaso Caselli

Fundamental properties of linear factor models


    Peter M. Kraus
    3 September 2024
================================================

§ ABSTRACT 
We propose misogyny detection as an Argumentative Reasoning task and we investigate the capacity of large language models (LLMs) to understand the implicit reasoning used to convey misogyny in both Italian and English. The central aim is to generate the missing reasoning link between a message and the implied meanings encoding the misogyny. 
Our study uses argumentation theory as a foundation to form a collection of prompts in both zero-shot and few-shot settings. These prompts integrate different techniques, including chain-of-thought reasoning and augmented knowledge. Our findings show that LLMs fall short on reasoning capabilities about misogynistic comments and that they mostly rely on their implicit knowledge derived from internalized common stereotypes about women to generate implied assumptions, rather than on inductive reasoning. 




§ INTRODUCTION





According to the 7^th Monitoring Round of the EU Code of Conduct on Countering Illegal Hate Speech Online,[<https://bit.ly/3yIRYWg>] Social Media are slowing down the removal of hateful content within 24 hours, dropping to 64% from 81% in 2021. The prevalence of hate speech phenomena has become a factor of polarization and pollution of the online sphere, creating hostile environments that perpetuate stereotypes and social injustice. 

Previous work on hate speech detection from the NLP community has contributed to definitions <cit.>, datasets <cit.>, and systems <cit.>. However, most of these contributions have focused (more or less consciously) on explicit forms of hate. Recently, there has been an increasing interest in the study of implicit realization of hate speech phenomena <cit.>.




Implicit hate speech is more elusive, difficult to detect, and often hidden under apparently innocuous language or indirect references. 
These subtleties present a significant challenge for automatic detection because they rely on underlying assumptions that are not explicitly stated. As illustrated in Figure <ref>, the  model[<https://huggingface.co/tum-nlp/bert-hateXplain>] correctly mark as hateful the explicit message (
    < g r a p h i c s >
), but it fails with the implicit ones (
    < g r a p h i c s >
).
To correctly spot 
the implicit message, the system would have to 
identify at least 

the implied assumptions that assume that “women aren't as capable as men.” and “women should be told what to do”.[Example and explanations extracted from <cit.>.] 


















In this contribution we investigate the abilities of large language models (LLMs) to correctly identify implicit hateful messages expressing misogyny in English and Italian. 
In particular, we explore how prompts informed by Toulmin's Argumentation Theory <cit.>

are effective in reconstructing the warrant needed to make the content of the messages explicit and thus facilitate their identification as hateful messages <cit.>. 


By prompting LLMs to generate such warrants, we further investigate whether the generated texts are comparable to those of human annotators, thus offering a fast and reliable solution to enrich hateful datasets with explanations and contributing to improve the generalization abilities of trained tools. 




We summarize our contributions as follows:

  
    
  * We present a novel formulation of implicit misogyny detection as an Argumentative Reasoning task, centered on reconstructing implicit assumptions in misogynous texts (<ref>).
    
  * We introduce the first dataset for implicit misogyny detection in Italian (<ref>).[All data will be released via a Data Sharing Agreement.] 
    
  * We carry out an extensive set of experiments with two state-of-the-art instruction-tuned LLMs ( and ) on English and Italian datasets (<ref>).
    
    
    

    
  * We conduct an in-depth qualitative analysis of the automatically generated implicit assumptions 
    against 300 human-generated ones (<ref>).
    
    



































§ RELATED WORK






Hate speech detection is a widely studied research area, covering different targets and linguistic aspects.
We discuss literature work on implicit misogyny detection with particular attention to contributions in reconstructing implicit content.



  
Implicit Hate Speech Detection

Hate Speech Detection is a 
popular research domain with more than 60 datasets covering distinct targets (e.g., women, LGBTIQ+ people, migrants) 
and forms of hate (e.g., sexism, racism, misogyny, homophobia)
in 25 languages, according to the Hate Speech Dataset Catalogue.[<https://hatespeechdata.com>]
In its early stages, but still predominant nowadays, research in this domain focused on developing datasets for detecting explicit cues of hate speech, like messages containing slurs or swear words <cit.>.
However, hate speech is often implicit, characterized by the presence of code language phenomena such as sarcasm, irony, metaphors, circumlocutions, and obfuscated terms, among others <cit.>.
For this reason, implicit hate speech detection has progressively gained momentum in recent years, and several efforts have been put into the development of datasets for this purpose <cit.>.
A relevant feature of these datasets is the presence of implied statements in free-text format, which contributes to explaining the content of hate speech messages.
While the use of these statements has been shown to have a positive effect on classification performance <cit.>, few efforts have been put in automatically generating such implied assumptions <cit.>. 
As <cit.> point out, current annotation schemes in this area present significant reasoning gaps between the claim and its implied meaning. 
Moreover, no effort has been made to evaluate widely adopted LLMs on their reasoning capabilities required to generate high-quality implied assumptions.

To the best of our knowledge, our work is the first study to propose an empirical evaluation of LLMs for implicit misogyny detection and the generation of explanations for Italian and English. 

Available datasets targeting misogyny in Italian <cit.> are highly biased toward explicit messages, with very few messages that qualify as implicit. 
To fill this gap, we have developed the first Italian dataset for this task, ImplicIT-Mis.
In our work, we define misogyny as a property of social environments where women perceived as violating patriarchal norms are “kept down” through hostile or benevolent reactions coming from men, other women, and social structures <cit.>, going beyond the simplistic definition of misogyny as hate against women.






  
Implied Assumptions Generation

The implied assumptions instantiate statements that are presupposed by the implicit hate speech message. This can be seen as the elicitation of implicit knowledge, corresponding to new content semantically implied by the original message <cit.>. 
Although limited, previous work on the generation of implied meanings —usually in the forms of explanations— has moved away from template-based methods <cit.> to the application of encoder-decoder or decoder-only models <cit.>. Generating explanations for implicit content poses    
multiple challenges concerning the quality of the generated texts, whose primary goal is to be reasonable and informative. 
Some approaches generate explanations by identifying pivotal concepts in texts and linking them through knowledge graphs <cit.>



More recently, the underlying concepts are generated by directly querying LLMs <cit.>.  
In this work, we follow the idea of using LLMs to identify the implied assumptions in the implicit messages, but rather than centering the reasoning process on identifying specific concepts, 

we formulate the problem as an Argumentative Reasoning task and apply Toulmin's Argumentation Theory <cit.>.



































    



§ MISOGYNY DETECTION AS A REASONING COMPREHENSION TASK




<cit.> introduced the Argument Reasoning Comprehension Task, which aims at identifying and reconstructing implicit warrants.
The concept of the warrant, as defined in Toulmin’s model of argumentation, plays a crucial role in bridging the gap between Reason and Claim. Warrants are usually presupposed and left implicit, thus, comprehending them requires language understanding, logic skills and commonsense knowledge <cit.>.
In the context of misogynistic discourse, the implicit warrant often consists of social bias, stereotypes and prejudices about women. Our hypothesis is that reconstructing these implicit warrants is essential for machines to understand and detect implicit misogynistic content, as it allows for the identification of the underlying prejudices that drive such discourse.
For instance, consider Fig. <ref>.






§ MISOGYNY DETECTION AS ARGUMENTATIVE REASONING UNDERSTANDING
 






The elusiveness of implicit hate speech is due to its ambiguity. 
Implicit messages could be understood as critiques, opinions, or statements (see Figure <ref>) rather than as hateful. 
Hate, in this case, is expressed by assuming social biases, stereotypes, and prejudices against a specific target, women in the case of misogyny.

The identification of these assumptions requires access to the reasoning process behind arguments and opinions.  



Argumentative Reasoning (AR) offers a solution. 

AR


relies on the notion of an argumentative model or scheme, i.e. a formal representation of arguments into intrinsic components and their underlying relations. It aims at explicating an argument through the identification of its constituent components and relations <cit.>. For instance, the Toulmin's AR model organizes arguments into fundamental elements such as claim, warrant, and reason. AR models have been successfully applied in many NLP tasks, from Argument Mining <cit.> to warrant and enthymeme reconstruction <cit.>, argumentative scheme inference <cit.>, and fallacy recognition <cit.>.





Grounded on previous work on AR in user-generated content <cit.>, we frame implicit misogyny detection as an AR task <cit.> based on the Toulmin's theory <cit.>, with the aim of developing more robust detection tools by explicitly describing the underlying reasoning process in these messages. More formally, let c be the claim associated to a given message 

and W = {w_1, …w_n } be a set of possible warrants, i.e., logical statement(s) that support c.
The model must generate an associated 
w and, based upon it,  
provide 
the requested classification
: whether the message is 
misogynous or not.

Figure <ref> graphically represents the approach described above. 

In this particular case, the generalization that women do not understand sport because it is stereotypically for men 
is what distinguishes a personal attack from a case of misogyny. 


While there have been efforts on evaluating LLMs in argumentative tasks, such as quality assessment <cit.>, component detection <cit.>, and argumentative linking <cit.>, the capability of LLMs for implicit argumentative reasoning has yet to be explored. To the best of our knowledge, our work is the first to assess LLMs on implicit misogyny through the lens of AR. 



























§ DATA






This section introduces the datasets used in our experiments. For Italian, the newly created ImplicIT-Mis corpus 
( <ref>). For English, SBIC+, an extended version of the social bias inference corpus <cit.> enriched with misogynous texts from Implicit Hate Corpus <cit.> ( <ref>).





 §.§ The ImplicIT-Mis Corpus



ImplicIT-Mis is a new manually collected and curated dataset for implicit misogyny detection in Italian. It consists of 1,120 Facebook comments as direct replies to either women-related news articles or posts on public pages of communities known to tolerate misogyny. 
An in-domain expert, who has been the target of misogyny, conducted the manual collection.
[The annotator is also an author of this paper.] This is in line with a participatory approach to NLP where the communities primarily harmed by specific forms of content are included in the development of datasets addressing these phenomena <cit.>. For each comment, we keep 
 source (either a newspaper or a Facebook page) and its context of occurrence (the news article or the main post). 
All instances in ImplicIT-Mis are misogynistic.


The collection period ran from November 2023 to January 2024. 
We selected 15 Facebook pages of news outlets covering the whole political spectrum as well as different levels of public outreach (national vs. local audiences), and 8 community pages. 
ImplicIT-Mis is organized around 104 source posts; 70% of the 1,120 messages are comments to news articles from two national newspapers (La Repubblica and il Messagero). The full overview 
is 
in Appendix <ref>. 
On average, each comment is 19 tokens long, with the longest having 392 tokens and the shortest only one. An exploration of the top-20 keywords, based on TF-IDF, indicates 
a lack of

slurs or taboo words, confirming the quality of our corpus for implicit misogyny.

ImpliciIT-Mis is enriched with one annotation layer targeting the implied assumptions, as defined in <ref>. A subset of 150 messages was annotated by three Italian native speakers who are master students in NLP. Each annotator has worked on 50 different messages. On average, annotators took 2 hours to complete the task. The annotation guidelines for the generation of the implied assumptions are in Appendix <ref>. 
We evaluated the annotators' implied assumptions against those of an expert (a Master student in gender studies and criminology). We used a subset of 75 sentences (25 from each annotator) and computed two metrics: 
BLEU <cit.> 
and BERTScore <cit.>. 

These measures offer insight into how similar the human written implied assumptions are. 
We have obtained a BLEU score of 0.437 and an F1-BERTScore of 0.685 by combining all annotations. 
As the scores indicated, our pool of annotators tends to write the implied assumption adopting different surface forms, 

but with a similar semantic content, as suggested by the F1-BERTScore. 
Although implied assumptions have to be inferred, and therefore, humans need to interpret the text, they tend to come to the same conclusions. 
In the final version of the data, all manually generated implied assumptions have been retained as valid, meaning that for 150 messages, we have a total of 225 implied assumptions.





















 §.§ SBIC+



SBIC+ is a dataset of 2,409 messages for implicit misogyny in English obtained by merging together 2,344 messages from SBIC and 65 from the Implicit Hate Corpus (IHC).

The social bias inference corpus (SBIC) <cit.> 
consists of

150k structured annotations of social media posts for exploring the subtle ways in which language can reflect and perpetuate social biases and stereotypes. It covers over 34k implications about a thousand demographic groups. SBIC is primarily composed of social media posts collected from platforms like Reddit and Gab, as well as websites known for hosting extreme views, such as Stormfront. 

The structured annotation approach implies that different annotation layers are available to annotators according to their answers. The annotation scheme is based on social science literature on pragmatics and politeness. We retain all messages whose annotation for the target group was “women” or “feminists” and were labelled as hateful. We further cleaned the data from instances labeled as targeting women but were actually targeting other categories, like gay males. We also filtered out all texts containing explicit identity-related slurs to keep only implicit instances. For each message, we also retained all associated “target stereotype” which correspond to the warrants. 

The Implicit Hate Corpus (IHC) <cit.> contains 6.4k implicitly hateful tweets, annotated for the target (e.g., race, religion, gender). The corpus comprises messages extracted from online hate groups and their followers on Twitter. 
Tweets were first annotated through crowdsourcing into explicit hate, implicit hate, or not hate.  Subsequently, two rounds of expert annotators enriched all implicit messages with categories from a newly developed taxonomy of hate, for the target demographic group, and the associated implied statement (i.e., the warrant in our framework). 
We have selected only tweets whose target demographic group was “women”.





§ EXPERIMENTAL SETUP



Our main goal is to evaluate the abilities of models to generate the implied assumptions for implicit misogynous messages. 
By doing so, we can also evaluate the implicit knowledge of LLMs, for instance, named entities or events mentioned in texts. 
If they are not known, it would be impossible to understand the misogynistic nature of such texts. 
Each batch of experiments aims to address two tasks: (i) the generation of the implied assumptions or warrants and (ii) the classification of the messages as misogynous or not.
Regarding (i), we experiment with two prompting strategies: instructing the model to reconstruct the implied assumptions (Assumption) and the implicit claim c and related warrants W (Toulmin). 
We address these tasks both in a zero-shot and in a few-shot setting. 

While implied assumptions are generally broader than warrants, warrants specifically bridge the reasoning gap between claims and evidence. In our prompts, implied assumptions and warrants appear quite similar. Nevertheless, the use of these terminologies may significantly impact the model's behavior due to its sensitivity to prompt phrasing, therefore we experiment with both.



We experiment with two state-of-the-art LLMs:  and .
[Refer to <https://huggingface.co/meta-llama/Meta-Llama-3-8B> and <https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2>]
For both, we select their instruction-tuned version. 
During preliminary experiments with 50 instances, we also tested Italian-specific LLMs, namely , , and . They were all unable to generate valid implied assumptions, so we discarded them.
We consider the following baselines: (i) finetuned encoder-based models; (ii) zero-shot classification with LLMs; and (iii) few-shot classification with LLMs without generating explanations.












  

The Llama3 series has several improvements over preceding versions, including a better tokenizer with a vocabulary of 128k tokens, extended training on 15T tokens, and grouped query attention for efficiency.
Around 5% of the pre-training data concerns more than 30 languages, including Italian.
All Llama3 models have undergone safety fine-tuning for safeguarding the generation process over harmful content. This could trigger instances of over-safety, with the model being unable to follow the instructions and thus failing to provide a valid answer for our task.







  
 
A competitive fine-tuned version of Llama2 using group-query attention, developed by MistralAI <cit.>.
In particular, the 7B version has been reported to obtain better performances when compared to  and .
While details about the fine-tuning data are lacking, in our experiments, we observe that the model is responsive to Italian prompts.
The instruct-based versions of the models do not present any moderation mechanism. We thus expect this model to avoid over-safety and always return an implied statement and a classification value.



 



 §.§ Prompting Techniques


Among recent prompting techniques, we selected Chain-of-Thought (CoT) and Knowledge Augmentation. CoT was chosen for its notable success in reasoning tasks <cit.>. 
On the other hand, Knowledge Augmentation has been observed to reduce hallucinations and enhance contextual depth in model prompts, facilitating the generation of sophisticated outputs beneficial for tasks requiring substantial domain knowledge and nuanced reasoning <cit.>.
Both techniques align with our goal of generating implicit components of arguments (implicit warrants) and support the construction of encoded warrant blocks. 
To the best of our knowledge, these techniques have not been used yet for a computational argumentation task, which makes them worth investigating. The full list of prompts can be found in Appendix <ref> and <ref>.

More in detail, CoT sequentially guides the model through a series of reasoning steps before arriving at a final answer or conclusion <cit.>. 
By following this structured approach, CoT prompts allow the identification of how the model's reasoning process influences its conclusions. This capability is particularly useful for reconstructing warrants that underlie the model's interpretations in our specific task. 


Knowledge-augmented prompting generates knowledge from an LLM and incorporates it as additional input for a task <cit.>. In our task, the generated knowledge serves as either the implied assumption or the warrant that we inject into the prompt to inform the classification.








§ RESULTS



We report two blocks of results: the first block focuses on classification of the messages. Since both the Italian and the English datasets contain only positive classes, we only report the Recall. The classification task offers an indirect evaluation on the goodness of the AR methods. The second block targets the generation of the implied assumptions/warrants. Considering the complexity and the pending issues related to the evaluation of automatically generated text <cit.>, we report the results using established automatic metrics (i.e. BERTScore and BLEU) as well as a manual validation on a subset of 300 messages (150 per language) ( <ref>). The overall evaluation procedure we have devised allows us to assess both the performance of the models' in detecting implicit misogyny and the alignment between LLMs and human annotators in generating reasoning-based explanations.











    



 §.§ Transformers




 §.§ Decoder-based Models




All answers from LLMs have undergone post-processing to evaluate them properly. Two main post-processing heuristics concern the treatment of the “refusal to provide an answer” (including the refusal to generate the warrants) and the “need of more context”. We considered both cases as if the messages were marked as not misogynous. While  tends to return refusals to answers, mostly due to the safeguard layer,   has a tendency towards indecisive answers requiring more context.  always provides an answer when applied to the Italian data. 
For completeness, Appendix <ref> includes the results considering these cases as correct.








 §.§ Classification Results



Table <ref> summarizes the results for the classification task. With few exceptions - mostly related to  - LLMs generally perform better than finetuned models. All few-shot experiments outperform their zero-shot counterpart, and  consistently performs better than . The best results are obtained by  with few-shot and no generation of either the implied statements or the warrants. However, for Italian, the  with the Toulmin warrant in few-shot achieves very competitive results (R=0.725). For English, on the other hand, the results are affected by the post-processing heuristics. Had we considered as correct the “refusal to answer cases”,  the best score for English would have resulted in  few-shot with implied assumption (R=0.913).


In all zero-shot settings, the prompt based on Toulmin's warrant outperforms the prompt based on implied assumptions. In the few-shot settings, in ImplicIT-Mis, we observe a dramatic increase when switching from implied assumptions to Toulmin's warrant, with a performance gain of 24 points. On the contrary, on English, the warrant-based prompt falls behind.




 §.§ Implied Assumptions and Warrants Generation
 


    



Table <ref> gives an overview of the evaluation using BERTScore and BLEU for the best models for English and Italian. While for SBIC+ every message has an associated explanation, for ImplicIT-Mis, only 150 messages present the implied assumptions. 
When  is asked to elaborate on the implied assumption in both zero- and few-shot settings, it does not follow the instruction, and only in 87 and 71 instances for Italian and English, respectively, generates a response.
In all the other cases, the model just answers the final question of whether it is misogynistic; therefore, we exclude them from the evaluation.
We also exclude all the results that do not reach at least a recall of 0.3 due to their low quality, as confirmed by manual inspection. 
All BERTScores in English are around 0.81-0.83, showing high similar content between the human-written texts and the answers generated by the models. Therefore, both the implied assumptions and the warrants are aligned with those written by humans. In Italian the scores drop to 0.57-0.60. In terms of BLEU scores, the highest scores for English are produced by  few-shots with warrants, which shows an alignment with humans in terms of word choices. For Italian the scores are much lower, probably because of many wrong translations and lack of Italian references which cause wrong inferences.





 §.§ Manual Validation
 

We further validate the generated implied assumptions and warrants by manually exploring a subset of 300 messages, 150 per language. 
For ImplicIT-Mis, we use the manually annotated instances, while we randomly extract 150 instances for SBIC+. 
We focus only on the best models:  few-shots warrant for ImplicIT-Mis and  few-shots implied assumptions for SBIC+.



Overall, we find that 35% of the generated warrants for ImplicIT-Mis are correct and 32% lead to a correct classification of the messages. For SBIC+, the percentage of valid implied assumptions leading to a correct classification is 50%, while correct implied assumptions leading to a wrong classification are 52%. 
However, in  Italian all the correctly predicted examples were actually predicted for the wrong reasons, while in English this happened 37% of the time. 
Therefore, we conclude that a correct explanation does not necessarily lead to a correct classification of misogyny, and this is always the case in the subsample we manually evaluated for Italian. This can be seen as an evidence that the model relies on their internalized knowledge and spurious correlations to address the task and shows no reasoning skills, since the Italian texts, being collected to address this task, requires much more reasoning to be understood.

We design a taxonomy to regroup all errors for both models.
We identify seven kinds of common errors in warrant and implied assumption generation. Table <ref> provides some examples. 

Notice that, although all error categories lead to wrong implied assumptions/warrants, we decide to keep a general “wrong inference” as a valid category for all the cases that do not fall under any other category or there is no evident reason.























  
Sarcasm/Irony 
This is a common error in English, due to the relatively high number of jokes in SBIC+.
In these cases, the LLMs fail to capture the sarcastic/ironic intended meaning of the message and go for a more literal interpretation. 




  
Metaphorical and Figurative Language.
This category indicates a failure to interpret another level of non-literal meaning. 
We have observed a much more frequent occurrence in Italian - also because many messages use figurative or metaphorical expressions. As observed by <cit.>, misogyny in Italian is highly metaphorical, especially with references to animals. 


In Italian, not identifying metaphors could also be attributable to translation errors since metaphors are cultural-dependent. 

This highlights the complexity of cross-lingual implicit HS detection, as also pointed out by <cit.>, since the translation of a term often does not carry the same implications as in the source language. 









  
Wrong Translations.
This is a category of errors that applies only to Italian.
It comprises errors due to wrong translations of messages or to a lack of understanding of non-standard language, such as dialects and jargon expressions. 







  
Opposite Intention. These errors could be considered an instance of LLM hallucinations <cit.>.
In these cases, the models completely misinterpret the message's content, resulting in generated implied assumptions that tend to support the message. 
These errors occur in both languages, with a slight preference for Italian.



  
Denial of Misogyny. 
This class of errors indicates a lack of connection between the generated implied assumptions and the answer of the model. 
In other words, the generated text is logical and correct as it clearly identifies the misogynous nature of the message.
However, the model classifies the texts as non-misogynous.



  
Lack of Reference

This kind 
of error has been observed only in Italian - mostly because very few English data contains direct reference to individuals. 
The errors in these cases are due to the model failing to identify the mentioned individuals (and their associated characteristics) that are used to trigger the misogynous content.
In the example we report in Table <ref>, the model does not recognize (Moana) Pozzi as a famous porn actress, thus resulting in a warrant that fails to capture the insulting nature of the message (i.e., the target being promiscuous).  




  
Wrong Inference

This is the largest class of errors in both languages. 
We observe that wrong inferences are mainly driven by spurious correlations and the activation of implicit knowledge. 























 §.§ How Reasoning Impact Classification


We further examine how reconstructed implicit assumptions relate to classification predictions for misogyny detection.


We inspect 50 instances for each language that are correctly classified by our best baseline (few-shot ) and that are subsequently misclassified when the model is asked to reason about them. 

We observe the tendency to reduce a gender-related problem to the whole category of human beings, minimizing the misogynistic nature of the statement.
For instance, in the Italian text “Oggi sei felice anoressica bugiarda” [Today you are happy anorexic liar], the model responds that “Anorexia is a negative condition. 3. Lying is a negative behavior. No, the text is not misogynist. Anorexia is a condition that affects both men and women.”. Although being true that it is a condition that affects both genres, in this case, it targets a woman which is clear in Italian by the gendered “anoressica”, instead of “anoressico” which would be used for men.
Adding the statement that it affects both genres is detrimental for the classification. 






















































    









§ CONCLUSION

We proposed the task of implicit misogyny detection under an Argumentative Reasoning perspective, since to understand implicit statements, one needs to reconstruct the missing link (the warrant) between the claim and the assumption. 
Our work highlights the complexity of such a task, which paves the way for hate speech detection as a proxy task to probe the reasoning abilities of LLMs.
Our prompt-based experiments show
that LLMs fail 68% and 50% of the time in generating implied assumptions in Italian and English respectively.

The poor relationship between wrongly generated explanations and correctly predicted classes shows
LLMs' over-reliance on their implicit knowledge and spurious correlations rather than reasoning skills.
Our results are consistent with <cit.>: prompting strategies that rely on implicit knowledge in LLMs often generate an incorrect classification when the generated knowledge (implied assumptions/warrants) is wrong, due to lack of references, reasoning skills, or understanding of non-standard language. 
Indeed, verifying the validity of the generated text before injecting it in the prompt in a human-in-the-loop approach would be a next step to undertake.

To conclude, our findings show that i) the performance of the classification task cannot be used as a proxy to guarantee the correctness of the implied assumption/warrant; ii) LLMs do not have the necessary reasoning abilities in order to understand highly implicit misogynistic statements. Therefore, models for hate-related natural language inference tasks should be improved.

One possible approach would be to inject external knowledge in the misogynous texts, in order to fill the gaps related to their lack of implicit knowledge. For instance, had the model known that Moana Pozzi was a porn actress, it would have probably inferred that when a person is compared to her, it is a derogatory way to address that woman.




§ LIMITATIONS

A limitation of our work is the integration of all generated knowledge (implied assumptions/warrants) and we do not evaluate them before using them to inform the classification task. This should be overcome with a human-in-the-loop approach that allows for the verification of the knowledge extracted by LLMs. We did not try to inject only the knowledge that led to a correct classification because of the low correlation between the generated implied statement and the class.
Another limitation is that for what concerns Llama, many examples in English trigger the safeguard, therefore the scores for Llama might not be realistic.



§ ETHICAL CONSIDERATIONS


Improving LLMs abilities to understand the implied meaning of messages with sensitive content is a case of potential risks related to dual use. Although our work has focused on assessing LLMs abilities in generating implied assumptions/warrants, we see the benefits and the detrimental effects. On the one hand, improving LLMs abilities to understand the implied meaning of sensitive message can further be used to improve the generation of counter-speech and the development of assistive tools for experts in this area. At the same time, the process can be inverted: malevolent agents can feed models with implied assumptions and generate hateful messages. We are aware of this issue, and we think our work offers the community an opportunity to understand limitations of LLMs that have a not minor societal impact. In addition to this, our work indicates the need to adopt different safeguard methods that are able to capture the core meaning of a message and grounded in different cultures.



















§ IMPLICIT-MIS SOURCES




 
 

Table <ref> shows statistics on the number of Facebook comments associated to each newspaper or Facebook community.






§ IMPLICIT-MIS ANNOTATION GUIDELINES FOR IMPLIED ASSUMPTIONS



Annotators were given the following guidelines for the identification of the implied statements in the ImplicIT-Mis dataset:

Read the following comments extracted from Facebook as a reaction to a news article or a post. The comments have been judged as being sexist or misogynistic by an expert. Your task is to provide an explanation for why the comment can be offensive towards. The explanation must correspond to the implied assumption in the comment. To help you in this task, imagine that your explanations should complete the sentence: “This comment is sexist because it implies that …”.
Follow the examples.
Le dovrebbero dare 2000 volts implica che la donna in questione dovrebbe essere uccisa.
Non dovremmo abbassare i nostri standard solo per assumere più donne implica che le donne sono meno qualificate degli uomini.
Quella mangia kinder bueno non meritava di vincere X-Factor implica che la donna in questione viene offesa in quanto grassa.
Bau bau implica che la donna in questione viene chiamata cagna.

Translated version:
She should be given 2000 volts implied that the targeted woman should be killed.
We shouldn't lower our standards just to hire more women implies that women are less qualified than men.
That Kinder Bueno eater did not deserve to win X-Factor implies that the targeted woman is called 'fat'.
Wof wof implies that the targeted woman is called 'cagna', translated as female dog, which is an epithet for b**ch.










§ ENGLISH PROMPTS




 
 







§ ITALIAN PROMPTS




 
 























§ ADDITIONAL CLASSIFICATION RESULTS



 
 

Table <ref> reports classification results when considering the refusal to answer due to model safeguard trigger to hateful content as misogynous.
In particular,  is the only affected model in our experiments.






 




