End User Authoring of Personalized Content Classifiers: Comparing Example Labeling, Rule Writing, and LLM Prompting
Leijie Wang Kathryn Yurechko Pranati Dani Quan Ze Chen Amy X. Zhang






End User Authoring of Personalized Content Classifiers]End User Authoring of Personalized Content Classifiers: Comparing Example Labeling, Rule Writing, and LLM Prompting










leijiew@cs.washington.edu

  University of Washington
  Seattle
  United States



kathryn.yurechko@oii.ox.ac.uk

  University of Oxford
  Oxford
  United Kingdom




pdani1@cs.washington.edu

  University of Washington
  Seattle
  United States



cqz@cs.washington.edu

  University of Washington
  Seattle
  United States



axz@cs.uw.edu

  University of Washington
  Seattle
  United States

























§ ABSTRACT 

Existing tools for laypeople to create personal classifiers often assume a motivated user working uninterrupted in a single, lengthy session.
However, users tend to engage with social media casually, with many short sessions on an ongoing, daily basis.
To make creating personal classifiers for content curation easier for such users, tools should support rapid initialization and iterative refinement.
In this work, we compare three strategies—(1) example labeling, (2) rule writing, and (3) large language model (LLM) prompting—for end users to build personal content classifiers.
From an experiment with 37 non-programmers tasked with creating personalized comment moderation filters, we found that with LLM prompting, participants reached 95% of peak performance in 5 minutes, beating other strategies due to higher recall, but all strategies struggled with iterative refinement. 
Despite LLM prompting's better performance, participants preferred different strategies in different contexts and, even when prompting, provided examples or wrote rule-like prompts, suggesting hybrid approaches.

















<ccs2012>
   <concept>
       <concept_id>10003120.10003130.10003233</concept_id>
       <concept_desc>Human-centered computing Collaborative and social computing systems and tools</concept_desc>
       <concept_significance>500</concept_significance>
       </concept>
 </ccs2012>


[500]Human-centered computing Collaborative and social computing systems and tools








  
[
    Amy X. Zhang
    September 5, 2024
=====================






§ INTRODUCTION

Today, internet users encounter more content than ever before, ranging from social media posts, blogs, and news articles to chat conversations and emails. To manage this overwhelming and sometimes unwanted information, online platforms provide automated curation systems to categorize, label, and moderate content. Some offer content moderation algorithms to remove harmful posts <cit.>, automatic classifiers to organize and prioritize emails <cit.>, or recommendation algorithms to discover interesting content <cit.>. But these systems are typically centralized and platform-wide, failing to accommodate the diverse preferences of individual users <cit.>.
To support end-user customization, researchers have examined and built a variety of specialized tools that enable users to author personalized content classifiers within social media <cit.>. They have also explored generic techniques for non-technical people to build their own text classifiers <cit.>. 


However, many tools for authoring personal classifiers on social media are designed for users with heightened motivation, such as community moderators <cit.> and high-profile content creators <cit.>. In focusing on these users, such tools neglect the usage patterns of general social media users, who spend significant time on social media over the long term but in many relatively short, fragmented sessions. They also have limited attention and motivation for cognitively demanding tasks <cit.>.
As a result, for many users to realistically participate in curation, tools for authoring classifiers should support rapid initialization. As end users oftentimes engage with internet content as a leisure activity, a successful tool should allow them to quickly and intuitively build an initial classifier with decent performance. In contrast, existing systems often demand that users maintain a high level of concentration for extended periods in this process <cit.>. For instance, some treat users as oracles who can continuously provide high-quality input, whereas others overwhelm users with an excessive amount of information <cit.>. 
In addition, curation tools should enable easy iteration to improve initial creations incrementally, as social media users may be more amenable to short tasks spread out over many sessions as opposed to s single lengthy task. Instead, existing systems often assume that users want to create highly performant custom classifiers in a single sitting <cit.>, for instance, requiring that users carefully debug their classifiers before their deployment. However, social media users naturally audit curation algorithms as they browse their feeds regularly <cit.>. Their preferences may also evolve gradually, necessitating continuous iteration <cit.>. Thus, a successful tool should enable users to make small adjustments over a period of time and ensure that each change results in the desired incremental improvement.


In this work, we use these system requirements to compare three prominent strategies for creating custom classifiers for content curation: (1) labeling examples for supervised learning, (2) writing and carrying out rules, and (3) prompting a large language model (LLM). 
Each of these techniques has its respective benefits and drawbacks for end-user classifier creation.
In interactive machine learning (IML) research, labeling examples is a popular strategy to customize a classifier. It is promising in the social media context due to its simplicity and ability to be easily divided into smaller tasks. However, even though pretrained word-embeddings <cit.> and active learning <cit.> reduce the number of labels needed to train a classifier, labeling examples can still be tedious and inefficient <cit.>, so this technique is typically not available for social media users to create personal classifiers from scratch.
In contrast, the most common strategy deployed on social platforms today is writing rules. Most platforms allow users to curate content through simple keywords <cit.>. AutoMod, the most widely used content moderation tool on Reddit <cit.>, further enables community moderators to write regular expressions to remove inappropriate posts <cit.>. Despite the transparency that rules afford, users often struggle to write rules that account for nuanced contexts and to further refine them <cit.>. 
Finally, with recent advances in LLMs, we additionally investigate the technique of customizing zero-shot or few-shot classifiers by prompting an LLM in natural language <cit.>. While LLMs have the potential to learn nuanced preferences quickly <cit.>, their effective use may require prompt engineering skills beyond the capabilities of many end users <cit.>.


We conducted a within-subjects lab experiment to evaluate these three strategies.
For each strategy, we implemented a content curation system with state-of-the-art features to ensure a fair comparison.
We then invited 37 non-technical social media users to build a personal classifier using each system. 
Each participant first manually labeled 100 comments sourced from YouTube videos on political topics to create a personal ground truth dataset, which allowed us to evaluate the performance of their classifiers. Then, in a randomized order, they used each of the three systems to create personal classifiers for removing unwanted YouTube comments, with a 15-minute time limit per system. 

We logged the actions participants took in each condition and tracked the performance of the classifiers as participants built them. After each condition, participants filled out a survey to report their subjective user experience.
Additionally, we conducted 13 semi-structured interviews with a subset of participants after all conditions were complete to better understand their challenges in communicating initial preferences and iterating on classifiers in different conditions.


We found that writing prompts generally allowed participants to create personal classifiers with higher performance more quickly, achieving a significantly higher recall and F_1 score than authoring rules and labeling examples throughout the 15-minute experiment period. 
However, participants struggled to iteratively refine their prompts to align LLMs with their nuanced preferences, as both recall and precision plateaued five minutes or earlier after starting.
This difficulty was most pronounced when users tried to teach LLMs to categorize specific phrases within certain concepts, such as classifying “fool” as offensive or “hell” as obscene. 
Even worse, the opaque nature and occasionally unpredictable behavior of LLMs left participants uncertain about which phrases LLMs might interpret differently. 
As a result, one-third of participants preferred labeling examples over writing prompts, while another third preferred authoring rules.
Some found labeling examples to be the easiest method for expressing their intuitive preferences, while they considered rules better for curating content about specific topics or events.
Participants even tried to deploy these two strategies when iterating on their prompts: they often added incorrectly classified examples directly into their prompts as few-shot examples or wrote prompts resembling rules. 

Our findings offer a roadmap for future end-user content curation systems and have broader implications for facilitating more efficient and effective communication between end users and classifiers. 
First, LLMs are a promising tool for enabling personalized content classifiers, with only five minutes of work needed to initialize a classifier with a better F_1 score than 15 minutes using traditional methods of labeling examples or writing rules.
However, we find that labeling examples requires less cognitive effort from users, and writing rules provides users with more control and transparency over their classifiers. 
Given the diversity of content curation scenarios and user preferences, future content curation systems should provide end users with more flexible strategies to create and iterate on their classifiers.
For instance, while writing prompts could be the default strategy given its superior overall performance, future hybrid systems could also suggest possible prompts based on example labels to help users who have difficulty articulating their intuitions, or allow rule authoring for preferences about specific topics or events.

Hybrid approaches such as these could bring together the benefits of all three strategies and facilitate more effective human-LLM collaboration.














§ RELATED WORK







 §.§ End User Customization of Content Curation

Content curation is an important process for internet users who seek to classify desired or undesired content on social media <cit.>, filter out harassment or spam from their email folders <cit.>, manage online chats <cit.>, find relevant news articles <cit.>, or otherwise categorize information that they encounter online. However, content curation tools are often centralized and platform-wide, leaving limited room for end user customization <cit.>. For example, one-size-fits-all moderation algorithms often fail to account for the diverse perceptions of toxicity across countries <cit.> and communities <cit.>. Similarly, some users want to categorize their emails into more fine-grained folders other than the default “Spam” or “Promotions” folders <cit.>. 
As a result, there have been growing calls to empower end users to customize content curation classifiers. For example, researchers have argued for personal content moderation tools that enable users to customize some aspects of their moderation preferences based on the content of posts uploaded by other users <cit.>. 
Users may wish to remove unwanted content from their feeds on platforms like Instagram, TikTok, and X/Twitter <cit.> or to moderate the comment sections of their YouTube videos or the chat channels of their Twitch streams <cit.>. In both cases, end users act as moderators that remove content according to their preferences.

Researchers and practitioners have experimented with offering custom content curation tools. However, these tools often struggle to navigate the trade-off between flexibility of customization and ease of use.
Some tools offer extensive customization but are too complex for casual users. For example, Reddit’s AutoMod <cit.> allows community moderators to write regular expressions in YAML to remove inappropriate posts, but only users with technical knowledge of regular expressions and programming syntax can fully utilize AutoMod's flexibility <cit.>. 
Conversely, other tools are more simplistic in design. Many of them involve pre-trained classifiers for platform-defined or researcher-defined concepts such as racism, misogyny, and political views, with users only able to adjust their sensitivity to each concept <cit.>. However, this approach fails when users have differing definitions of a concept or wish to classify content with respect to a concept that is not provided <cit.>.
Recently, research has shown that LLMs can outperform state-of-the-art classifiers in detecting toxic content with natural language prompts <cit.>. But LLMs are not exempt from the customization versus ease of use trade-off: when applied toward more complex and nuanced curation preferences that go beyond simple hate speech or toxicity filtering, LLMs sometimes perform even worse than a coin toss <cit.>. In this work, we compare common content curation techniques in terms of this trade-off through an experiment using the case of personal content moderation, with the eventual goal of designing a system that offers both customization and ease of use for a broad range of end users. 



 §.§ Strategies that Support End Users in Customizing Classifiers for Content Curation


Despite the ubiquitous use of machine learning (ML) algorithms in various domains, only a small group of people with technical expertise possess the skills to develop these algorithms. Interactive machine learning (IML) seeks to democratize ML training with humans-in-the-loop, enabling non-experts to participate through “rapid, focused, and incremental model updates” <cit.>. IML underlies many efforts at developing end user systems to build classifiers and operates in three critical phases  <cit.>: (1) Users examine a model’s outputs across a set of examples; (2) Users then offer feedback to the model, guiding its learning in the desired direction; and (3) Users assess the model's overall performance and decide whether to conclude the training process or to offer the model more feedback.


At the heart of these phases are teaching vocabularies: the frameworks through which end users structure their feedback  <cit.>, including assigning labels, selecting features, specifying parameters, or indicating error preferences <cit.>. Recently, prompts have emerged as another promising way for end users to communicate with algorithms (i.e., LLM-based models). Through the lens of teaching vocabularies, we characterize existing content curation systems by three primary strategies that they leverage to support end user customization: labels, features or rules, and prompts. In the following, we discuss each strategy and its relationship to content curation.



  §.§.§ Labels


Many IML systems consider end users to be oracles who provide correct labels. Techniques like pre-trained word embeddings <cit.> and active learning <cit.> are often used to minimize the number of labels required to effectively train a ML algorithm <cit.>. An active learning algorithm iteratively picks examples for labeling from an unlabeled dataset through query sampling strategies, such as uncertainty sampling <cit.> and query-by-committee <cit.>. Previous studies have shown that people find labeling effective for communicating with opaque, “black-box’’ algorithms <cit.>. However, researchers have noted that end users might find the repetitive labeling of data tedious and non-transparent <cit.>. There is also a risk of users applying labels inconsistently due to their evolving preferences <cit.>.



  §.§.§ Features or Rules


In response to the aforementioned criticisms of the labeling approach, a body of research focuses on feature-level human input <cit.>. Here, the term “feature’’ refers to an attribute of a data instance that ML algorithms use to predict its class label. In comparison, “rules” are often logical statements that directly map to a class label without relying on ML algorithms. In systems that support technical users to build a text classifier via feature-level input, features and rules often take the same form: the mentions of keywords. By incorporating features or rules into different algorithm architectures, these systems can offer varying degrees of transparency.
Rules can offer the highest transparency by directly classifying text without using any ML models. 
Features, on the other hand, can be incorporated into transparent ML algorithms, such as Decision Trees, Naive Bayes algorithms <cit.>, and Support Vector Machines <cit.>. 
In addition, because end users might produce noisy and inconsistent features <cit.>, user-generated features can also be used to label data. Weak supervision algorithms, for example, then learn from these “soft labels’’ <cit.>. However, these indirect approaches sacrifice the transparency and control that users often assume with feature-level input. For this reason, many systems designed for end users often integrate features with transparent algorithms to enhance users' mental models of such systems <cit.>.

Although empirical results regarding whether feature-level input from end users improves performance have been mixed <cit.>, researchers are generally in agreement that feature-level input requires fewer user actions to achieve comparable results to example labeling, and that it could produce models more closely aligned with an individual’s needs or domain knowledge. However, features can be seen as too granular and therefore not generalizable enough, posing a challenge for end users in creating effective classifiers <cit.>.



  §.§.§ Prompts


LLMs have exploded in popularity due to their proficiency in a broad range of natural language processing tasks, such as sentiment analysis and machine translation. Compared to features, natural language prompts can convey richer user guidance to models and thus promote greater generalizability across various contexts <cit.>. Rather than having to train a new model for every custom task, users can simply customize LLMs by feeding them prompts at run time. Such an ability to recognize the desired task on-the-fly is called in-context learning (ICL) <cit.>.

Developing effective prompts for ICL is crucial to leveraging LLMs in content curation <cit.>. To date, the most common patterns for prompting are zero-shot or few-shot prompts. Zero-shot prompts give instructions for a task without any specific examples for training <cit.>. Research indicates that the performance of zero-shot prompts can be enhanced by iteratively refining task instructions <cit.> or breaking down tasks into simpler subtasks <cit.>. Alternatively, few-shot prompts incorporate a few input-output examples to showcase the desired pattern to which LLMs should adhere <cit.>. The quality of few-shot prompts relies heavily on the selected examples  <cit.>. While few-shot prompts generally outperform zero-shot prompts, the simplicity of composing natural language instructions without the need for selecting examples makes zero-shot prompting an attractive option for end users. Even so, various challenges impede end users in creating and refining their prompts. For example, even a slight adjustment to a prompt's format or ordering of examples can greatly influence a model’s performance <cit.>. In addition, users often find it difficult to evaluate the clarity of their instructions to LLMs and to align their perspectives with those of LLMs, leading to incorrect or unexpected outputs <cit.>.



 §.§ How Existing IML Systems Overlook End User Needs in Content Curation


While researchers and practitioners have built many systems using labels, features, and prompts, they often make assumptions about end users that do not translate well into the context of content curation. 

First, these systems often require end users to spend hours of dedicated time creating their custom classifiers <cit.>. However, users typically engage with social media as a leisure activity <cit.>. 
For example, a recent survey notes that the top two reasons why people use social media are to keep in touch with friends and to fill spare time <cit.>. Research also suggests that internet users often rely on default settings, despite acknowledging the benefits of customization controls <cit.>. Ultimately, users are only willing to invest effort in creating custom classifiers if there are proportional improvements to their content feeds <cit.>.

Additionally, existing IML systems require users to be highly dedicated to creating their custom classifiers. 
For instance, some systems treat users as infallible experts who can continuously provide high-quality labels or rules <cit.>, while others present an overwhelming amount of information intended to facilitate the teaching process <cit.>. 


Even though weak supervision algorithms can learn from noisy user annotations, they often require more annotations to compensate for their low quality <cit.>. Such an expectation of high commitment could thus intimidate users—who are likely to be casually using classifiers <cit.>, accessing them on mobile devices, or using them in between other activities and with distractions <cit.>—from using these systems at all.

Finally, IML systems often assume that users want to create highly performant classifiers at their inception <cit.>, but users of content curation tools prefer to iteratively refine their classifiers <cit.>. 
For example, instead of carefully considering all possible mistakes when initially creating their classifiers, social media users naturally audit their deployed classifiers as they browse content feeds for entertainment <cit.>. Members of online communities also tend to help community moderators collect mistakes via user reports <cit.>. 
This on-the-go monitoring signals the need for content curation classifiers to adapt to constant distribution shifts and fluid curation preferences: a new political event or video theme could spark community discussions that were unobserved or nonexistent when a user initially developed their classifier <cit.>. Users' curation preferences may also evolve over time, sometimes even depending on their moods <cit.>. However, iterating a classifier remains challenging in IML research <cit.>. Traditional ML algorithms often require users to provide sufficient new training data or relabel existing training data to reflect updated information. While several systems enable users to update a model via rules <cit.> or natural language descriptions <cit.> by adjusting the weights of training data, it remains to be explored how these methods could be effectively applied in the context of content curation.

















































§ METHODS

Drawing inspiration from research in interactive machine learning and in-context learning, we identified three primary strategies, or “teaching vocabularies,” through which users can convey their personal preferences to algorithms: labeling examples, authoring rules, and writing prompts. 
Each teaching vocabulary can be leveraged most effectively by a corresponding backend architecture: black-box ML algorithms, transparent ML algorithms, and LLMs respectively. From the perspective of users, each teaching vocabulary provides different levels of control and requires varying degrees of effort to learn and master. 
Corresponding to these three teaching vocabularies, we developed three representative systems enabling end users to create binary text classifiers, using the scenario of personalized content moderation for YouTube comments. Each system is either a popular or potentially popular tool for end users to customize moderation algorithms. 


We conducted a within-subjects experiment with 37 non-programmer participants to comparatively evaluate these three personal moderation systems for everyday users. Participants used each of the systems—in a randomized order—to build a classifier within a fixed amount of time, resulting in three experimental conditions. Throughout the duration of each condition, we collected the performance of participants' classifiers every half-minute in order to understand how well each system enables rapid initialization and iterative improvement over time.
After each condition, we collected participants' self-reported usability ratings of the system in that condition to understand their experiences and challenges that they encountered. For 13 participants, we additionally conducted a semi-structured interview after the experiment was over to understand their challenges in creating and iterating personal classifiers.
We further describe our system implementations and experiment design below.








 §.§ Experiment Systems

Through pilot studies (detailed in Appendix A), we iteratively implemented three systems corresponding to the three primary teaching vocabularies: Label System (labeling examples to train a supervised model), Rule System (writing executable rules), and Prompt System (writing prompts for an LLM). To ensure the validity of our experiment, we established the following criteria in our system implementations.

First, we restricted the form of user input in each system to its designated teaching vocabulary. 
While we acknowledge that an ideal personal moderation system might accept multiple forms of user input, we note that allowing varied types of user input in a single system could obscure the comparison between the three teaching vocabularies in our analysis.
For instance, in a label-based system, allowing users to influence algorithm predictions by highlighting keywords in examples could blur the lines between teaching through labels and teaching through rules (since keywords act as rule indicators).
Similarly, if users are allowed to instruct an LLM to enumerate all possible offensive words in a rule-based system, then users would essentially use both rules and prompts to interact with the algorithm.

Second, we ensured that similar functionalities were integrated across all systems so that no condition had an unfair advantage, provided that such functionalities did not compromise the user experience. For instance, both the Rule System and the Prompt System enable users to interactively view how their systems label examples from the training dataset. However, we did not include this feature in the Label System because our pilot studies indicated that displaying system predictions on examples during the active learning process tended to confuse rather than assist end users.

Finally, we equipped each system with state-of-the-art features to assist end users in creating their moderation classifiers. For example, the Rule System suggests synonyms for phrases that users have already input, and the Prompt System helps rephrase users' prompts. 
We chose to implement features that involve relatively trivial and already established improvements to the basic authoring interaction to improve performance or usability <cit.>.
Note that this approach does not conflict with our first criterion, as we only enhance user input and do not permit multiple forms of user input.
Thus, our three systems present our attempt to provide the best versions of each strategy while keeping each strategy distinct and functionalities similar for a fair comparison. 
In Figure <ref>, we summarize the features we chose to implement for each system.
In the following, we discuss the detailed implementation of each system, highlighting how the systems are optimized to support users in the context of personal moderation.




  §.§.§ Label System: Train a “Black-box” ML Algorithm by Labeling Examples.

With the Label System, users label example comments to create a personal moderation classifier. We experimented with various models using the Jigsaw toxicity dataset <cit.> and opted for a combination of Sentence Transformer + Naive Bayes, which had the highest overall performance.

We incorporated active learning to enhance our system, specifically employing uncertainty sampling <cit.>. After a user labels a set of examples, we train the algorithm on these labels and calculate the predicted positive probabilities for the remaining examples in the training dataset. We subsequently sample examples with the highest label uncertainty. In the context of Naive Bayes, examples with prediction scores near 0.5 are deemed the most uncertain. The frontend web interface, depicted in Figure <ref>, allows users to label examples selected by active learning and then train the backend algorithms.











  §.§.§ Rule System: Create a Transparent Algorithm by Authoring Rules


In implementing the Rule System, we drew inspiration from AutoMod, which allows community moderators to create automated scripts for detecting rule violations, such as profanity or external links. Recognizing that AutoMod’s complexity may deter end users from creating personal moderation classifiers, we used our pilot studies to develop a more user-friendly Rule System tailored to individual users' moderation needs (see Appendix A). The system filters comments that match any constructed rule, which represents a category of texts that the user wants to remove from their content feed. Figure <ref> presents an example of rules that users can create with our Rule System. Authoring rules is typically an iterative process in which users review examples, refine their rules, and examine the effect of their rules on those examples <cit.>. Therefore, we present examples from the training dataset on the side, as seen in Figure <ref>, so that users can interactively review the effect of their rules and then adjust their rules accordingly.






  §.§.§ Prompt System: Communicate with LLMs by Writing Prompts.




Like the Rule System, the Prompt System features a panel of instructions (prompts in this case) and a panel of examples. Figure <ref> provides an example of a prompt as a category of unwanted comments. We refrain from asking LLMs to generate explanations for their predictions on individual examples because prior studies indicate that such explanations can be inaccurate, potentially confusing users rather than helping them develop clear mental models <cit.>. 

When developing the prediction algorithm, we needed to determine the optimal balance between the quality of LLM predictions and the time users must wait for predictions.
Through our pilot studies, we decided to prioritize low response times while maintaining sufficient prediction quality to reflect realistic deployment settings (see Appendix A).
Our prediction algorithm operates as follows. It combines each user-created prompt with a predefined system prompt and then requests predictions from LLMs in batches of 10 comments.
Since LLMs process each prompt independently, we can tell users which prompt leads to the removal of an example, thus offering more explainability than aggregating all prompts into a single query. We also introduced a caching mechanism—only re-querying the LLM for prompts that have been modified since their last evaluation—to minimize unnecessary requests and therefore reduce users' waiting times. Throughout our implementation, we used OpenAI's  model.






 §.§ Experiment Datasets

For our experiment dataset, we crawled all comments of three videos about gun control policies from a YouTube channel called The Young Turks: an online news show with 5.78 million subscribers that covers political topics from a left-leaning perspective. We refrained from using existing toxicity datasets <cit.> because they were often sampled from various online platforms and communities to train more generic toxicity classifiers. Their examples relate to diverse contexts and are difficult for participants to understand. Hence, we concentrated on three videos regarding the same topic to simulate a real-world personal moderation setting more accurately.
We selected the channel The Young Turks for various reasons. First, given our plan to recruit university students, we chose the familiar and opinion-provoking topic of political news over niche interest group content. Additionally, to elicit more opinionated reactions, we opted for a channel known for its more polarizing and occasionally controversial content. Lastly, we selected a channel without active comment moderation to ensure the presence of potentially toxic ones in our dataset.

We gathered over 5,000 comments from the three selected videos. To simplify the moderation task, we excluded comments that were responses to others. We also filtered out comments that were too brief or excessively lengthy, as they tended to be either irrelevant or cumbersome to read. Next, we used the Perspective API to assess the toxicity level of each comment. We labeled a comment as toxic if its toxicity score exceeded 0.7, following recommendations from prior research <cit.>. Although participants' moderation preferences can differ greatly, we aimed to balance the dataset so that nearly half of the comments would be toxic as determined by Perspective API. This process resulted in a balanced dataset of 800 comments. We randomly divided our dataset into a training dataset and a test dataset of 100 examples for each participant. The training dataset was used to help participants create their classifiers, whereas the test dataset was labeled by participants and used to evaluate their created classifiers.



  §.§.§ Recruitment and Participants

We recruited 37 non-programmer participants by advertising a call for participation on mailing lists of non-Computer Science departments at two major U.S.-based universities. We only selected participants who self-reported having little knowledge of programming and algorithms. There were 27 females, 9 males, and one participant who preferred not to disclose their gender. Most participants were pursuing their bachelor's degrees except four pursuing more advanced degrees.
Regarding political stance, 16 participants identified as liberal, 12 as moderate, and the remaining 9 were evenly distributed among “very liberal,” “conservative,” and “prefer not to disclose” groups. In terms of generative AI usage, 14 participants used it at least every few weeks, 14 participants every few months, and 9 participants rarely.

To gather qualitative data on participants' experiences with the systems, we conducted 13 individual user studies via Zoom. Additionally, we held 7 in-person workshops that did not include semi-structured interviews. Each workshop had between three to five participants, with all participants in a given workshop using the three systems in the same sequence. The individual user studies averaged 129 minutes in length, while the in-person workshops lasted about 100 minutes. Individuals were compensated with a $40 gift card for their participation. In our analysis, we denote participants who attended individual sessions as P1–P13 and those who did not as W1–W24.




 §.§ Experimental Design




  §.§.§ Study Design and Procedure

We employed a within-subjects design with the three experiment systems described above. 
The final experiment protocol was designed iteratively through five pilot experiments to ensure its effectiveness. 
This study was reviewed by our university IRB and deemed exempt. 

Stage 1: Study Onboarding. We started the experiment by briefing the participants and warning the possibility of encountering profanity and hate speech. We emphasized that participants could stop the experiment whenever they wanted, and we gained their explicit consent before proceeding. We then asked participants to imagine themselves as YouTube content creators whose videos on gun control policies had gone viral and were flooded with comments. They were then invited to set up automated classifiers to remove unwanted content based on their personal preferences. To gather a wide range of user preferences in our study, we assured participants that we would not judge their moderation preferences but instead focus on how well the three systems could align with their preferences.

Stage 2: Ground Truth Labeling. Subsequently, participants were asked to label 100 comments as “Keep” or “Remove” as the test dataset.
Prior research suggests that users might label data inconsistently, which harms the training of downstream ML algorithms <cit.>.
Hence, we emphasized that participants should make labeling decisions consistently and that, if they change their criteria, they should revise their previous decisions. 
This test dataset would later be used to evaluate the performance of the classifiers that participants created.
We scheduled the ground truth labeling before the classifier creation so that participants could familiarize themselves with content moderation and this dataset of YouTube comments. While exposure to the test dataset beforehand might bias the classifier creation process, we argue that participants already had moderation preferences in mind and simply conveyed them when labeling their test datasets. 


Stage 3: Creating Personalized Classifiers. We then asked participants to create classifiers using experiment systems in a randomized order. We used a counterbalanced design to counter any potential learning or fatigue effects. 
The underlying process for all three experiment conditions remained consistent. Each condition lasted 25 minutes in total. Participants first spent five minutes engaging with tutorial slides and the corresponding system. They were asked to try all of the system's functionalities and were encouraged to ask any questions. 


Then, in each condition, participants were given 15 minutes to create classifiers. 
In particular, the waiting time for backend computations did not count toward the 15-minute duration. This affected two systems: the Prompt System supported by a generative language model, and the Label System, which takes time to calculate the most uncertain examples for the next batch.

After 15 minutes, participants spent three minutes examining the overall performance and individual predictions of their created classifiers on their test datasets. Following this, participants reported their subjective experiences in a survey, which we will discuss in detail in <ref>. Once participants completed all three conditions, we conducted a final survey to collect participants' preferred systems for content moderation and their rationales.

Stage 4: Semi-structured Interviews (for Individual User Studies Only). 
At the end of the experiment, we asked participants a series of questions to understand the challenges that they encountered in creating and iterating on their classifiers. Examples of these questions include “What do you like and dislike about writing prompts?” and “Can you easily understand why a prompt classifier makes its decisions?”
We also inquired about their preferred systems for various content curation scenarios. 






  §.§.§ Evaluation Measures

We consider the following three measures to evaluate the systems in our experiment.

Classification Performance. To determine which system could learn user preferences with the highest performance, we evaluated the accuracy, precision, recall, and F_1 score of the final systems participants developed. 
We adopted several performance metrics because individuals prioritize different metrics for content moderation algorithms <cit.>. Additionally, since participants' ground truth labels on the test dataset were not always balanced, accuracy might not reflect a system's true ability to distinguish between positive and negative examples.



Creation Speed. To assess which system could enable participants to develop a performant classifier most rapidly, we logged each participant's classifier every 30 seconds throughout the classifier creation period and calculated the performance of each snapshot. 
The performance of each classifier at early intervals serves as an indicator of its creation speed. In particular, we selected 5 minutes and 10 minutes as two representative intervals.

Ease of Creation. We documented 23 different types of user interactions, including asking for synonym suggestions, loading additional examples, and applying classifiers to examples.  These logged actions indicated the usefulness of the implemented features and how participants approached classifier creation with each system. 
We also gathered participants' subjective perceptions using a five-point Likert scale (from Strongly Disagree to Strongly Agree) for each condition:


    
  * Subjective Workload. We adopted four applicable items from the NASA-TLX survey <cit.> regarding mental demand, temporal demand, effort, and feelings of stress. 

    
  * System Usability. We adopted all four items from the Usability Metric for User Experience (UMUX) survey <cit.> to measure system usability. We chose the UMUX over the System Usability Scale (SUS) to reduce the overall number of questions that participants had to answer.

    
  * Understanding. To understand whether participants had a clear mental model of each system, we evaluated both participants' global understanding of a system and their local understanding of individual predictions <cit.>. For global understandings, participants were asked to rate the statement  “I felt that I had a good understanding of how my classifier works.” For local understanding, we asked participants to explain the predictions of one false positive and one false negative randomly selected from the test dataset.




 §.§ Data Analysis



  §.§.§ Quantitative Modeling
 Quantitative results were analyzed using a linear mixed-effects (LME) model, where experiment systems were treated as a fixed effect and participants as a random effect. The dependent variables in our model included classification performance, creation speed, and various subjective measures of user experience. We did not include the order in which participants used the three systems as another fixed effect, as no significant differences were observed for this variable. For each dependent variable, we calculated pairwise differences between the three experiment systems. Finally, we conducted a sanity check to confirm that participants exhibited diverse moderation preferences.  We found that half of the ground truth comments had, at most, a 75% majority consensus.



  §.§.§ Qualitative Coding
 Our qualitative data comprised semi-structured interview data and responses to open-ended questions in surveys. We employed a reflexive thematic analysis approach <cit.> to explore participants' experiences and challenges in creating personalized classifiers with each system. 
Reflexive thematic analysis has been widely used in HCI research to understand users’ experiences and views, as well as factors that influence particular phenomena or processes <cit.>. During data collection, the first author took detailed debriefing notes after each interview to document emerging themes. The authors then collectively reviewed the debrief notes and discussed themes in weekly group meetings. Recordings were automatically transcribed into text. The first author then open-coded the data on a line-by-line basis, and the remaining authors reviewed the transcripts and added codes. Over 300 codes were generated from the open-coding process. The authors clustered the open codes into high-level themes in a codebook and iteratively improved the codebook through discussion. Some examples of codes are Examples: Failed to understand preferences, Rules: Transparency, and Prompts: Idiosyncratic behaviors. Finally, the authors applied the codes to the data to complete the thematic analysis.





§ RESULTS






 §.§ What strategies enabled participants to create high-performing classifiers quickly?




  §.§.§ The Prompt System resulted in the best final performance after 15 minutes, although both the Prompt System and the Rule System had the highest precision

Figure <ref> compares classification performances across the three systems over 15 minutes, with each line's endpoint representing the final performance. Table <ref> presents the pairwise differences in the final performances among the three systems. We observe no significant differences in accuracy across systems. In terms of precision, the Rule System showed significantly higher precision than the Label System (Est.Diff = 0.097, p  < 0.05) but did not differ significantly from the Prompt System. However, the Prompt System demonstrated significantly higher recall compared to both the Label System (Est.Diff = 0.205, p  < 0.001) and the Rule System (Est.Diff = 0.232, p  < 0.001). Consequently, the Prompt System also achieved a significantly higher F_1 score than the other two systems (Est.Diff compared to the Rule System = 0.105, p  < 0.01; Est.Diff compared to the Label System = 0.141, p < 0.001). 
We focus on F_1 as a performance metric of interest over accuracy due to the unbalanced nature of participants' labels.
Finally, when examining how the final performance of each system varied across participants, we found that the Prompt System showed the least variance, regardless of the performance metric used. This suggests that the Prompt System is less susceptible to individual variances than the other systems.



  §.§.§ The Prompt System reached 95% of its peak performance within 5 minutes on average, with gains due to rapid initial improvements in recall

Table <ref> and Table <ref> present the pairwise differences in classifier performance at 5 and 10 minutes respectively. Consistent with our earlier findings, the Prompt System demonstrated significantly higher recall and F_1 scores than the other two systems. Notably, its recall at 5 minutes had already surpassed the final recall of the other two systems.
Meanwhile, the Prompt System's recall exhibited a distinct pattern: a rapid increase in the first five minutes, followed by a steady climb in the subsequent five minutes, and a plateau in the remaining time. In contrast, the Rule System showed a consistent increase in recall throughout the entire 15-minute period, whereas the Label System showed a less noticeably upward trend in recall over time.
On average, participants reached 95% of their peak recall by 7 minutes, and that of their peak F_1  score by 5 minutes. We also observed that participants made fewer changes to their prompts as time went on, while they continued actively labeling examples or curating keywords for their rules in the other two systems. These findings suggest that writing prompts can facilitate rapid initialization but may be less effective in supporting iterative improvements. 




  §.§.§ Users struggled to improve the precision of all three systems over time.


The precision of the Prompt System and the Rule System remained comparable throughout the experiment. 
While the Prompt System showed significantly higher precision than the Label System at 5 minutes (Est.Diff = 0.131, p < 0.05), the difference became non-significant at 10 minutes (Est.Diff = 0.082, p = 0.090). In contrast, while initially comparable at 5 minutes (Est.Diff = 0.100, p = 0.06), the Rule System had a significantly higher precision than the Label System at 10 minutes (Est.Diff = 0.099, p < 0.05) and at the end (Est.Diff = 0.097, p < 0.05).

While both the Prompt System and the Rule System showed notable improvements in recall by the end of the experiment, their precision either plateaued or even decreased over time. 
For instance, the Rule System experienced a gradual decrease in precision during the first 5 minutes, followed by a steady state for the remaining 10 minutes. The Prompt System, despite a slight increase in precision in the initial 2 minutes, subsequently experienced a minor decline before reaching a plateau. 
These observations align with participant behaviors: participants tended to add more categories of unwanted content rather than increase the specificity of existing rules or prompts during the task because the latter was often considered more cognitively demanding.
Although the Label System demonstrated a rapid increase in precision during the first five minutes, it began with such low precision that its final precision was still below that of the other two systems.





  §.§.§ Summary

In summary, the Prompt System enabled end users to create custom moderation classifiers with superior performance more rapidly compared to the other two systems. 
In terms of the precision-recall trade-off, its higher performance was primarily driven by its highest recall, as its precision remained comparable to the Rule System. 
Temporally, we observed that the Prompt System facilitated rapid classifier initialization during the early stage but faced challenges in supporting further iteration, as evidenced by its plateaued precision and recall afterward.
In contrast, both the Rule System and the Label System showed steadily increasing trends in their recall and F_1 scores throughout the experiment, despite being constrained by their significantly lower initial values.















 §.§ What strategies did participants find easiest for communicating their preferences?








  §.§.§ When participants had ill-defined but intuitive preferences, they found the Label System to be easiest

Unlike professional moderators, social media users often lack the time or opportunity to articulate their moderation preferences clearly <cit.>. Instead, their moderation decisions are often a result of their intuitive feelings about the content they encounter. P4 pointed out that “sometimes it might be the entire sentence that I don't want to see, or maybe it's just about a word...It could also be the feeling of the comment. You don't feel right about it.” 
Some participants also lacked “a holistic view about what were the alarming comments” (P2), since new content continuously flooded their social media feeds. 
Labeling examples thus provided them with a natural way to express their intuitive preferences to the algorithm, whereas writing prompts or authoring rules forced them to distill their intuitive preferences into high-level patterns.
Participants tended to trust algorithms to infer their preferences from their labels more than they trusted themselves to accurately convey their preferences, especially in time- and attention-constrained settings of personal moderation.
As P2 explained, “I would actually trust the algorithm finding trends more [than] me looking at everything and trying to build my own...especially with my lack of experience and lack of time.” This trust made participants more willing to offload the task of summarizing high-level criteria to the algorithm, despite this requiring them to label many examples.

In contrast, translating their intuitive preferences into rules or prompts required participants to iteratively refine their rules or prompts to better convey their intuitions, a process that participants largely considered to be mentally demanding. P10 described their process of writing prompts in detail: “In the beginning, I have an idea of what I do not want to see and then I would submit that. But [AI] wouldn't know what I was talking about, so I had to change my wording to match what AI would understand.” For some participants, this process nudged them to actively reflect on their intuitions about what kinds of content they wanted to moderate, but for others, it was simply too mentally taxing. P2 expressed the latter sentiment: “It was almost more tiring to write prompts [than label examples], as you need to think about each one but then apply it, and then see one little thing you could have fixed, and then reapply it.” This is part of the reason why participants rated the Label System as significantly less frustrating and demanding than the other two systems in Figure <ref>. 







  §.§.§ When participants had well-defined and general preferences, they found the Prompt System to be easiest

Some participants could clearly define their general preferences regarding themes like violence or hate speech before even engaging with a classifier.
Unlike authoring rules, they could directly translate those preferences into prompts without the need to curate a comprehensive list of keywords. 
P6 explained this advantage of writing prompts: “For rules, if there are 1,000 words I don't want to see, I have to list all 1,000 words, [whereas for ChatGPT], I only need to think of 10 words among those 1,000 words, and then ChatGPT itself understands that this guy does not want this kind of words.” 
This advantage is especially pronounced because some preferences are hard to capture with keywords. P10 highlighted their struggle to author rules for such preferences: “It is easier to think about specific words for removing profane content, but then for violence, I didn't know what words specifically to say [for such content].”
Even if participants could identify keywords to indicate their preferences, they sometimes hesitated to add them because these keywords could also be used in neutral or benign contexts and thus adding them could result in many false positives.

Additionally, writing prompts simplified the articulation of complex preferences, which would otherwise require complex rule structures.
For instance, P11 tried to create a rule to “remove personal attacks against an organization.” Even though they managed to compile two lists of phrases—one for “personal attacks” and another for “organizations”—they found that the conjunction of these two concepts in a rule structure did not accurately capture the intended relationship between them.

While rules provided participants with a structured, albeit somewhat cumbersome, way to map their pre-established preferences, labeling examples was considered the least straightforward method for expressing such preferences. 
P13 communicated their frustration with the inability to explicitly state their preferences through labeling: “There are too many decisions to make. Yeah, they're small decisions of yes or no. But usually, I don't have to keep creating 50 small decisions to indicate my personal preferences. Why am I going through hundreds of these when I just want to focus on saying my preferences?”
Participants' senses of frustration further increased when they could not find sufficient relevant examples to label as a way of indicating their preferences, as described by P10: “I feel like I have this criterion [remove violence calls] in my head. But there is nowhere I can label these examples.”
This issue is especially severe when users try to develop a classifier for a new community with a limited number of examples available.
Finally, participants were concerned about whether algorithms would accurately learn their high-level preferences from labeled examples. P13 worried, “What if all the comments happened to have [the word] Texas, so the AI just picks up those all related to Texas?”

Despite the prompts' advantages over the other approaches, participants still found it challenging to describe concepts outside of LLMs' existing knowledge. For example, LLMs lacked awareness of the context, such as the video or post to which comments were attached. P7 wanted to “remove irrelevant comments” and wished for “ChatGPT to be familiar with the content of the original post.” Additionally, LLMs often struggled with more complex concepts such as misinformation or conspiracy theories. W14 experienced this limitation: “I wanted to remove this comment because it contains obvious disinformation about the city of Portland Oregon, but my filter did not understand my definition of obvious disinformation.”






  §.§.§ When participants had well-defined preferences regarding specific topics or events, they found the Rule System to be easiest


While rules as a keyword-based approach may overlook the broader context of comments, they can effectively adhere to preferences that can be captured by a list of keywords. Although people could write prompts that resemble rules in the Prompt System, these prompts could still remove comments that included similar concepts. P8 described the difference between authoring rules and writing prompts along this dimension: “[By authoring rules], you might get to really specific thing, and lost the big picture. But then [by writing prompts], you have just these larger generalizations. You know you can't get down to the nitty-gritty and identify truly exactly what phrases you want to remove.” In addition, rules are particularly useful for hiding content about specific events. As P1 explained, “I would only use the rule system if there was a particular topic that was very distressing for me, like a bombing somewhere. If I were personally affected by that and didn't want to hear anything about that, I would just create a short list of relevant words. ” 



 §.§ What strategies were easiest for iterating on classifiers after initialization?




  §.§.§ Transparency in the Rule System and the Prompt System enabled participants to pinpoint problems but not always be able to fix them, whereas the Label System offered less transparency and little opportunities for targeted iteration.


Participants had a clearer understanding of why the Rule System made incorrect predictions than the other two systems. Such transparency helped participants improve their classifiers in some cases, like when they forgot to include spelling variants or similar phrases to those that they explicitly included in their rules. But in other cases, transparency did not lead to iterative improvements because rules could not accommodate contextual variances, as documented in prior research <cit.>. Our survey and interviews suggest that participants had a slightly better understanding of why their classifiers made mistakes for the Prompt System than for the Label System (Figure <ref>). Participants found that they could at least review and reflect on problematic prompts in the Prompt System, whereas they could not pinpoint specific labels that led to classification mistakes in the Label System. P2 noted this advantage: “The prompt system was more transparent in the sense that it highlights specific guidelines you have set up. You could then change them and see what they do.” 

For the Label System, participants had only a vague understanding of why their classifiers made incorrect predictions, reflecting previous research on how end users develop folk theories to interpret content curation algorithms <cit.>. Moreover, it offered few opportunities for targeted iteration, which were particularly useful as participants found it easier to point out what the classifier should learn from its mistake. This highlights the need for more flexible approaches at different stages of classifier creation, as recent studies advocated <cit.> 
In the absence of such features in the current system, participants could only label more examples similar to these mistakes in the hope of providing targeted feedback.



  §.§.§ Users of the Prompt System struggled to refine their initial prompts due to human-LLM misalignment and LLMs' unpredictable behaviors.


While participants could quickly build classifiers with decent performance using the Prompt System, they found it challenging to incorporate more nuances when iterating on their prompts. Participants rated it as comparably difficult to correct with the Rule System (Figure <ref>).
Many participants described a disconnect in how humans and LLMs perceive the same prompts. They observed that “descriptions could be very subjective” (P4) and asked, “What has [the LLM] previously been trained on? For words they labeled as extreme, I might not think as extreme” (P13). Participants also suggested that LLMs might not fully understand participants' experiences and personalities from their prompts and might therefore fail to execute their preferences faithfully. Consequently, many noticed that LLMs seemed to interpret their preferences broadly, causing LLMs to have trouble distinguishing different degrees of a concept. For instance, P13 noted that “[LLMs] have the problem of differentiating between just minor attacks on someone's character [such as calling someone a fool] and ones that are strong and can be perceived as harm.” Similarly, P8 struggled to teach LLMs to “differentiate between threats and just using the word `shoot' or `kill.'” Articulating the fine-grained differences between concepts proved too difficult for many participants.
While they could easily teach LLMs what was clearly good or bad, aligning LLMs with their moderation decisions for borderline cases remained a challenge. 

In addition, participants sometimes faced confusion in response to the peculiar behaviors of LLMs, leaving them unsure of how to further improve their prompts. For instance, P3 was perplexed by their prompt classifier's removal of the comment “Jessica is the best,” explaining, “I'm honestly not sure why the filter removed this content. I wrote something about removing comments with harmful descriptions, so maybe it thought that this description was harmful in some way.” Sometimes, participants were surprised that the Prompt System did not remove comments that they included as few-shot examples. P1 frustratedly said, “With ChatGPT, I would directly copy and paste a comment that I didn't want, and it still wouldn't remove it.” 
Even though the performances of participants' prompt classifiers were comparable to the other two systems' performances, such incomprehensible behaviors undermined their confidence in using LLMs for personal moderation. 



  §.§.§ Even when using the Prompt System, participants iterated using Rule System- and Label System-like strategies


Surprisingly, to further refine their prompts to align with their nuanced preferences, participants often learned from strategies of authoring rules or labeling examples. Several participants opted to avoid using high-level descriptions in their prompts and instead wrote prompts that resembled rules, such as “Remove texts that refer to people as stupid, dumb, idiots [a list of words].” In this way, they expected that LLMs could still catch various spelling variants of these words but would not generalize more broadly beyond the word list.
Alternatively, when unsure how to describe nuances in prompts, many participants resorted to adding representative positive or negative examples directly into their prompts as few-shot examples. In this process, they were essentially labeling examples rather than writing prompts. Four participants even included more than five examples in their prompts. 
W6 remarked: “I liked that I could copy and paste certain examples in ChatGPT. It made it easier to capture ideas that specific words do not allude to but are alluded to by unique comments.” 
However, some participants also recognized the limitations of this approach. As P13 noted, “I think [adding examples to prompts] is good up to a certain extent. If you know there are comments you don't want, it's great and...also gives you more nuances. But sometimes when a lot of these comments are extremely similar, [the LLM] just gets confused.”











 §.§ Which strategies did participants prefer overall?




  §.§.§ There was no clear preference for a particular system across all participants; instead, different systems were favored based on varying use cases and individual needs.

We asked participants to rank the three systems regarding their preferences for using them in real-life content moderation and to explain their rationale for these rankings. 
Figure <ref> illustrates the distribution of participants' rankings. 
Surprisingly, despite growing enthusiasm for using LLMs to moderate content, a comparable number of participants still preferred labeling examples or authoring rules to create custom classifiers for content moderation. While the Prompt System was rated first most often, it was also rated third most often, suggesting that it was a polarizing choice. 
The Wilcoxon Signed-Rank test showed no significant differences between any of the systems. 

By examining data from our semi-structured interviews and open-ended responses from the final survey, we identified several dimensions that contribute to this variance in preferences. 
They highlight the principal differences across the three systems, the diversity of individual preferences, and the variety of content curation scenarios for end users.


    
  * 

Whether users want explicit articulation of their preferences. Individual preferences for content curation are nuanced and subject to change. During the experiment, many participants tended to modify their criteria after reviewing more examples. 

As a result, if participants had explicitly stated their preferences in rules or prompts, they would have to continually reflect on their preferences and update their classifiers accordingly. In comparison, the Label System's algorithms had the potential to learn these subtle changes in preferences from a continuous flux of labels. As P12 explained, “Preferences change over time, and we may not explicitly realize that. So you may accidentally remove some things if your preferences change but you have already explicitly set them in stone. ” However, some participants appreciated the chance to actively reflect on and articulate their preferences through high-level rules or prompts, despite the additional effort required.




  * Whether users can tolerate reviewing many toxic examples. In our study, participants reviewed fewer comments to develop an effective classifier using the Prompt System compared to the other two systems. On average, participants reviewed 133 and 127 examples for the Label System and the Rule System respectively, whereas only 53 examples were needed for the Prompt System. The need to review numerous toxic examples could be distressing, particularly for those creating classifiers to block such content. Whether people are willing to undertake such an emotional toll can depend on their sense of responsibility in specific moderation scenarios. As P8 stated, “If I were moderating for my own sake and not for a community, I would probably just focus more on the ease of creating a filter so that I don't have to subject myself to these potentially offensive comments.” 



  * Whether users prioritize precision over recall. Participants demonstrated varied approaches to balancing precision and recall. Some favored precision because they were more concerned about the risk of removing benign comments than approving  unwanted content. P12 described their rationale as follows: “My preference is to limit the number [of comments] that accidentally gets removed. I don't want to fall into any echo chamber. Even if there are really crazy people saying really crazy things, they make me frustrated, but I still should know about it.”  Other participants favored recall, showing a stronger inclination to remove toxic comments, even if it meant mistakenly removing benign content. P13 expressed this preference: “Personally, too specific is worse than the too broad. If the filter keeps certain content I really don't want to see, that's far worse than if I didn't get an extra post. Because of how much content there already is, I'll just get others on my feed anyway.” For these participants, the high recall of the Prompt System is even advantageous during the creation stage. Since LLMs could collect the majority of potentially unwanted examples given initial criteria, participants could focus on reviewing these examples to increase the precision of their prompts.



  * To what extent users value transparency and controllability. An important distinction between the Rule System and the other two systems lies in transparency. 
As P10 described, “I like rules more because I know what's going into them. But with the prompts, you don't know what's going on. While the prompting is better at catching things, I just like the transparency of rules more.”
The Rule System helps users not only understand why exactly a comment is removed; it also enables moderators and users to have clear expectations about which comments will be removed. For users who moderate for a community, this sense of controllability could enable them to be more accountable for their moderation decisions. For example, P9 shared, “I think there's like a fine line between filtering or censoring. I think it's important to know why certain comments or text was filtered.”






































§ DISCUSSION


In this work, we compared three prominent strategies for creating custom classifiers, focusing on their support for rapid initialization and easy iteration. Our experiments revealed that writing prompts generally enabled participants to create custom classifiers with the highest performance most quickly. However, this approach also had shortcomings, such as challenges in further iteration and lack of transparency. In this section, we discuss how incorporating labeling examples and authoring rules could help mitigate these problems, and how the unique user needs of diverse content curation applications demand different kinds of hybrid systems.



 §.§ Hybrid Approaches to Personalized Classifier Creation

Existing tools often fail to accommodate the diversity and fluidity of user needs in content curation. While writing prompts generally proves to be an effective way to convey user preferences to classifiers, our experiment indicates the potential to incorporate labeling examples and authoring rules into future content curation systems, thereby providing end users with more flexible strategies to create and iterate on their classifiers. 

Labeling examples can facilitate easier prompt bootstrapping. 
Our findings indicate that when users have ill-defined but intuitive preferences, they prefer labeling examples over writing prompts to convey those preferences. 
However, with traditional ML-based algorithms, users often complain that they need to label numerous examples to express their preferences and that there is a lack of transparency regarding what the algorithm learned from their labels.
We envision that users can provide a few representative examples for LLMs, which help infer and suggest potential preferences.
Labeling examples could also help users easily iterate on their prompts. In our experiments, when users tried to refine their prompts, they often focused on a few misclassified examples and adjusted their prompts until LLMs made correct predictions.
Given the mental load of such interactive iteration for many participants, we propose developing an automated prompt chain where LLMs automatically refine prompts based on a few corner-case examples users curate and label.

Authoring rules offers users a stronger sense of transparency and controllability than writing prompts. Our experiments indicate that rules could easily capture preferences involving specific topics or events.
Therefore, instead of completely replacing rules with prompts, content curation tools should allow users to choose between rules and prompts based on their preferences.
Future research should also investigate ways to increase the transparency of LLM predictions. 
In our implementation of the Prompt System, we asked LLMs to process each prompt independently rather than aggregating all prompts into a single query. 
This approach is akin to connecting prompts in a rule structure, thus offering more transparency. 
Similarly, researchers could introduce more transparency into LLM predictions by decomposing a complex prompt into a series of conditions and querying LLMs separately. 
This method allows users to understand which specific conditions might be causing mistakes <cit.>. 



 §.§ Supporting Different Content Curation Applications

As discussed, content curation scenarios can range from curating personal feeds to managing community content. Individuals could also have intuitive preferences or gradually develop more well-defined preferences from initial intuitions. Additionally, individuals vary greatly in many dimensions, such as tolerance of undesirable content and the precision-recall trade-off. Future research should therefore only apply our findings after closely examining the unique needs of their contexts.
For example, individuals looking to remove undesirable content from their feeds may prefer prompt-based systems over existing rule filters due to their ease of use. In contrast, prompt-based systems might only be desirable for community moderators if they offer comparable transparency to existing rule systems, as moderators are more accountable for their moderation decisions <cit.>. 
Additionally, despite their algorithmic similarities, curating desired content requires different systems than removing undesirable content, since users tend to prioritize recall over precision during active exploration and have more fluid and intuitive preferences in this case than in content moderation <cit.>. Future work should thus explore how to combine the high recall of writing prompts and the ease of labeling examples into one system for this use case.

In this work, we investigate supporting users to create personal classifiers from scratch but social media users might want to modify existing classifiers. Some platforms have already offered similar but simplified functionalities, allowing users to adjust the sensitivity for predefined concepts like racism and misogyny <cit.>. Compared to a platform-centric definition of these concepts, end users might feel more comfortable customizing classifiers created by friends or family members who share similar interests and moderation preferences <cit.>. Our findings also advocate for more flexible customization options beyond simply adjusting thresholds <cit.>. For example, we envision users being able to merge labeled data, fork and edit rules, or share prompts.

While we focus on content-based curation in this work, end users also frequently curate content based on metadata. 
For instance, they might want to see all posts from certain accounts regardless of content or may prefer the latest content when curating time-sensitive information. Here, the challenge lies in helping users curate and summarize information for decision-making, such as evaluating whether a user is worth following or identifying when content becomes interesting. Future research should explore how to support end users in communicating preferences that involve conflicting non-textual and textual information (e.g., what if someone I follow posts unwanted content?).
Additionally, while our findings on binary classifiers could be generalized to a categorical classifier, they cannot be easily extended to regression algorithms. 
Such algorithms are still valuable for users who want to sort content in their feeds, as in the context of content recommendation <cit.>. 
Common content recommendation systems often learn from example-level feedback but suffer from similar issues identified in our experiments, such as few opportunities for explicit feedback and lack of transparency <cit.>. Future work should investigate the potential of using rules or prompts in this space.



Finally, while there is growing interest in more cost-effective LLMs <cit.>, the cost of deploying an LLM-based content curation system remains a concern given the vast amount of online content produced daily. 
For instance, on a platform level, there are 500 million tweets sent every day <cit.>, and on a user level, a popular YouTube video can gather thousands of comments.
Although techniques like multi-step reasoning <cit.> or self-consistency <cit.> can enhance the performance of LLMs on complex user preferences, they often require more computational resources. Future research should investigate leveraging techniques such as LLM cascading <cit.> or model distillation <cit.> to maintain performance while reducing costs. For example, cheaper models could be used for less complex or less important user preferences, whereas LLMs could be distilled into lightweight classifiers for heavy users.





§ LIMITATIONS

While our experiment protocol was developed iteratively through pilot studies, it still has limitations. First, we determined our three experiment systems by mapping each strategy to their most common backend algorithms (supervised learning algorithms for labels, transparent algorithms for rules, and LLMs for prompts). As a result, we did not have the chance to test other less common combinations in our experiments, such as supervised learning algorithms for rules (i.e., a Snorkel-like system <cit.>).
Second, even though we explicitly encouraged participants to label the test dataset consistently, exposure to more examples from the training dataset during classifier creation might alter their criteria. To prevent the experiments from being excessively long, we limited participants to labeling only 100 comments that comprised our test dataset, which might affect our evaluation's accuracy. 

Besides, our participant pool of undergraduates did not represent the diverse population of internet users, and our study involved a relatively small number of participants. Future work should conduct larger-scale experiments to validate our findings.  
Fourth, our study was conducted solely in English and focused on political content, limiting its applicability to other languages and types of content. Since LLMs might have different performances for other languages or content types <cit.>, future research should validate and extend our findings.
Finally, while personal content moderation as a case study illustrates the primary user needs for content curation tools, we acknowledge the diversity of content curation scenarios. Future work should examine whether there are distinct user needs in other scenarios before applying our findings broadly.






§ CONCLUSION

To communicate feedback to an algorithmic system, end users can primarily leverage three strategies: labeling examples, authoring rules, and writing prompts. 
Which strategy is the most useful for end users to customize a classifier for content curation?
Using personal content moderation as a case study, we conducted a within-subjects experiment with 37 non-programmer participants to compare these three prominent strategies for creating custom classifiers.
We found that writing prompts generally allowed participants to create personal classifiers with higher performance more quickly, but not without shortcomings. 
While prompts could effectively communicate users' well-defined and general preferences, participants preferred labeling examples to convey their ill-defined but intuitive preferences and authoring rules for their preferences about specific topics or events.
Moreover, participants found it challenging to refine their prompts to communicate their nuanced preferences iteratively. Consequently, they often directly added misclassified examples as few-shot examples or attempted to write rule-like prompts.
Building on top of our findings, we envision a hybrid approach to custom classifier creation: users could label examples to bootstrap and iterate on their prompt classifiers, while decomposing a complex preference into a rule structure could provide users with greater transparency.





















ACM-Reference-Format








§ PILOT STUDIES


We conducted pilot studies with our lab members and 5 non-programmers to iteratively implement our three systems. We included non-programmers in these studies to guarantee that our interfaces would be user-friendly and accessible to all potential users.



 §.§ Label System


Since both the Rule System and the Prompt System allowed users to review the performance of their filters during the creating process, we tested real-time feedback for the Label System in our pilot studies. Specifically, when users were labeling examples from the new batch of active learning, they could review how the filter that was trained on previous labels would predict on each example. We experimented with displaying this information immediately after users labeled an example and after they labeled all examples in a batch. However,  our pilot studies suggested that such features were more confusing than helpful for end-users, so we excluded them from the Label System for our experiments.



 §.§ Rule System


We used our pilot studies to iteratively develop a more user-friendly Rule System than AutoMod that is tailored to the moderation needs of individual users. Specifically, we made the following changes to adapt the AutoMod system for personal moderation.


    
  * AutoMod permits a variety of conditions in a rule such as “include,” “exclude,” “start with,” or even regular expressions. However, regular expressions might be excessively difficult for end-users, and certain conditions, like “start with,” may not be as beneficial for filtering unwanted content. To simplify the rule creation process, our system restricts end-users to using only “include” and “exclude” conditions. We call the exclude condition an “exception” because users might be confused by a complicated rule like “remove a unit that includes this word but excludes that word.”

    
  * While AutoMod allows an unlimited number of conditions per rule, this complexity can be overwhelming and not particularly useful for end-users in expressing their preferences. Therefore, we limit users to having at most two “include” conditions and one “exclude” condition per rule to maintain simplicity and clarity.
    
    
  * AutoMod offers a range of actions for each rule, including approving and removing caught texts. In contrast, our system is designed solely for creating rules that remove texts. This design choice is based on observations that navigating between “approve”/“remove” actions and “include”/“exclude” conditions can be confusing for end-users.

    
  * Prior research has indicated that users often struggle to account for all potential spelling variants that could bypass word filters <cit.>. To reduce the mental load of brainstorming synonyms for phrases, we integrated a “similar phrases” feature that leverages the LLM to suggest phrases that are similar to existing ones provided by users. Additionally, our system offers an option to detect the following spelling variants. We implemented the detection of the first three types of variants by generating generalized regular expressions for each phrase. For nouns and verbs, we first identify the part-of-speech tag of the word <cit.>, then search for its plural form if it is a noun or its various tenses if it is a verb <cit.>.
    
     
         
  * phrases with repeated letters, such as “coooool” for “cool”

         
  * phrases with mixed uppercase and lowercase letters, such as “Cool” for “cool”

         
  * phrases where letters are replaced with visually similar characters, such as “co0l” for “cool”

         
  * singular or plural forms of nouns, such as “apples” for “apple”

         
  * different tenses of verbs, such as “found” for “find”
     




 §.§ Prompt System


We evaluated two potential methods for the Prompt System. First, we put all user-generated prompts into a single system prompt template and only queried the LLM once for the overall prediction. This approach compromised the quality of LLM predictions since the LLM sometimes skipped a few examples or completely ignored certain prompts. 
Second, we queried the LLM multiple times for each prompt and determined the final prediction based on the majority. This quality-minded approach often required users to wait more than 5 minutes for LLM predictions and thus discouraged users from interactively testing their filters during the creation process. 

Some may advise prioritizing the quality of LLM predictions during the filter creation process in order to evaluate the full potential of LLMs in personal moderation. However, content moderation algorithms in deployment often need to analyze vast quantities of comments per day. Prioritizing quality without regard for efficiency can therefore lead to impractical and costly operations. To ensure that users accurately evaluate how well their filters would work once deployed, it is essential that, during the filter creation process, users engage with the algorithm under conditions that mirror actual deployment settings. Thus, we opted to prioritize a low response time while ensuring that prediction quality was still sufficient in order to mirror realistic deployment settings.
Ultimately, we used the following system prompt:









