# Beyond Dark Patterns: A Concept-Based Framework for Software Design Ethics

Evan Caragay  Massachusetts Institute of Technology32 Vassar
StCambridgeMassachusettsUSA02139 0009-0005-7286-5403<EMAIL_ADDRESS>,
Katherine Xiong  Massachusetts Institute of Technology32 Vassar
StCambridgeMassachusettsUSA02139 0000-0003-0856-2123<EMAIL_ADDRESS>,
Jonathan Zong  Massachusetts Institute of Technology32 Vassar
StCambridgeMassachusettsUSA02139 0000-0003-4811-4624<EMAIL_ADDRESS>and  Daniel
Jackson  Massachusetts Institute of TechnologyCambridgeMassachusettsUSA02139
0000-0003-4864-078X<EMAIL_ADDRESS>
(2018; 20 February 2007; 12 March 2009; 5 June 2009)

###### Abstract.

We present a framework of dark patterns grounded in user expectations. In
contrast to prior approaches that treat design techniques as inherently either
good or bad, we analyze mismatched user expectations for application behavior
using concepts—reusable units of functionality that users encounter across
applications. We define a design as dark when its concepts violate users’
expectations, and benefit the provider of the application at the user’s
expense. Though user expectations can differ, leading to subjective
perceptions of the ethics of an interface, users tend to develop common
expectations as they encounter the same concepts repeatedly across multiple
applications. This reuse results in users having shared expectations of
concept functionality, which we can record as standard concepts. Through case
studies, we illustrate how concept analysis helps designers identify, compare,
and resolve dark patterns. We suggest a shift away from dark pattern
taxonomies toward more systematic, actionable accounts of interface design
ethics.

dark patterns, user expectations, concepts, design, ethics, procedural and
substantive theory

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation emai; June 03–05, 2018; Woodstock,
NY††booktitle: Woodstock ’18: ACM Symposium on Neural Gaze Detection, June
03–05, 2018, Woodstock, NY††price: 15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs:
Human-centered computing HCI theory, concepts and models††ccs: Social and
professional topics

##  1\. Introduction

When users notice unethical design in applications, they may feel annoyed or
frustrated—but they also feel resigned. There is a sense that unethical design
is so rampant that users just have to live with it and that efforts to push
back are a waste of time (Maier and Harr, 2020). To characterize these
frustrating designs, researchers have developed the notion of “dark patterns”
to highlight widely occurring designs that are misleading or unethical.
Currently, researchers primarily view dark patterns through extensive
taxonomies or thematic categories of manipulation techniques (Brignull et al.,
2023; Gray, 2022; Mathur et al., 2021), which treat designs as inherently good
or bad. These approaches usefully provide shared language for describing
unambiguously problematic designs, but because a particular design is either
included in a taxonomy or not, their ability to analyze unethical design is
limited. They provide little help explaining why some designs feel worse than
others, or evaluating ambiguous cases that seem unethical, but are not overtly
manipulative or deceptive.

To address this limitation, we present a framework for understanding dark
patterns defined in terms of user expectations and values, using concepts
(Jackson, 2021)—reusable units of functionality that users encounter across
applications—to analyze expectations for application behavior. Concepts were
first introduced by Jackson as a tool for designers to systemically reason
about applications in terms of user experience, similar to how design patterns
(Gamma et al., 1994) help programmers reason about the programmer experience
of a code base. Examples of concepts for an online store might include
Shopping Cart, Order, Catalog, etc.

Our framework recognizes three versions of a concept that are present in every
design: the observed concept, or the concept that is communicated to a given
user via the user interface; the implemented concept, or the functionality
programmed into the application; and the expected concept, or the concept the
user expects to interact with based on their interactions with applications in
the past. This categorization lets users of our framework systematically
analyze users’ subjective perceptions and expectations for a given interface
design. In our framework, a dark pattern arises for a given user when the
observed or implemented concept conflicts with the expected concept, such that
the violation benefits the system’s designer or owner to the detriment of the
user. This two-part definition accounts for design choices that cause
mismatches in a user’s expectations, and also accounts for how those choices
express values that might conflict with a user’s values.

Although user expectations are not universal, users do form shared
expectations that develop over time as they encounter the same concepts
repeatedly across applications. We introduce the concept catalog as a method
for formally defining standard expectations for concepts and for how concepts
are mapped onto user interface elements. Standard concepts help make the
question of subjective user expectations more tractable for designers and
policymakers by introducing a way to build consensus around common
expectations. Designers who implement standard concepts can reduce the
likelihood of their applications being perceived as unethical, and can help
more firmly establish positive standards for design as commonplace
expectations.

Our definition of dark patterns is informed by procedural and substantive
theories of ethics. Procedural theories focus on “standardized, repeatable
steps” while substantive theories focus on the normative values of different
stakeholders and the use of power in resolving those differences (Zong and
Matias, 2022). Concept analysis and standard concept catalogs provide
designers a procedural method to analyze whether a design is meeting users’
expectations. Simultaneously, they are a tool to help designers break down
substantive, values-driven questions, moving from vague questions about the
whole design to specific, targeted questions about design elements

We evaluate our framework’s descriptive power, finding it is able to cover
existing taxonomies while adding additional insight into why known dark
patterns are dark. We also find that concept catalogs can accurately and
concisely describe functionality consistent across popular applications on the
web. We evaluate our framework’s evaluative and generative power through case
studies, examining three different interfaces that are similar in terms of
functionality, but the ethics of which are interpreted differently by users.
We demonstrate how concept analysis helps designers not only identify features
of designs that contribute to negative user perceptions, but also gives them
the tools to understand differences in user perceptions and develop actionable
solutions to better align the design with expectations.

##  2\. Related Work

###  2.1. Dark Patterns and Deceptive Design

Harry Brignull, who introduced the term dark patterns in 2010, defines them as
“tricks used in websites and apps that make you do things that you didn’t mean
to” (Brignull, 2010; Brignull et al., 2023). Since then, researchers have
created additional definitions of dark patterns, often centered around the
idea of deception. Some focus on designers’ use of “deceptive functionality
that is not in the user’s best interest” (Gray et al., 2018), while others
focus on violations of individual autonomy that involve “coercing, steering,
or deceiving users into making decisions that, if fully informed and capable
of selecting alternatives, they might not make” (Mathur et al., 2019).

Researchers apply these definitions to examine instances of context-specific
dark patterns, typically creating a new taxonomy of dark patterns for every
new domain that is analyzed (Council, 2018; Soe et al., 2020; Zagal et al.,
2013; Gunawan et al., 2021; Mathur et al., 2019; Di Geronimo et al., 2020;
Kowalczyk et al., 2023; Mildner et al., 2023; Hidaka et al., 2023).
Practitioners and regulatory bodies have also developed numerous dark patterns
taxonomies for enforcement purposes. Brignull lists sixteen types of deceptive
design on his website (Brignull et al., 2023), and legislative bodies
including the US Federal Trade Commission, UK Competition and Markets
Authority, and EU Commission have all published work (noa, 2022a, b;
Commission et al., 2022) listing egregious dark patterns and providing
guidelines to help companies avoid them.

More recent scholarship has attempted to move the literature away from
taxonomies and toward highlighting the specific routes of influence that dark
designs use. In 2021, Mathur et al. posited that dark patterns arise from
modifying the user’s choice architecture, either through manipulating the
available information or modifying the available choices (Mathur et al.,
2021). Similarly, Gray et al. tried to unify regulatory and academic
taxonomies of dark patterns by creating a three-level ontology to characterize
all dark patterns as a high-level, meso-level, or low-level pattern (Gray et
al., 2023).

Most proposed solutions for combating dark patterns involve three main
stakeholders: designers, policymakers, and users. For application designers,
proposed interventions include integrating ethics into designer education
(Maier and Harr, 2020), providing ethical guidelines around what nudges to
user behavior are acceptable (Renaud and Zimmermann, 2018), and giving
specific design recommendations (Utz et al., 2019). For policymakers, research
has proposed better enforcement of existing laws and providing tools to
automatically identify dark patterns (Mathur et al., 2019). Finally, for
users, Zagal et al. proposed “manipulation literacy” to help with identifying
dark patterns (Zagal et al., 2013). Rossi and Bongard-Blanchy outline the full
space of possible interventions, recommending that researchers consider “dark
pattern attributes in combination with the user perspective” (Rossi and
Bongard-Blanchy, 2021).

###  2.2. Ethics of Interface Design

Scholars across HCI and design have long studied how design shapes the values
that are expressed in software, and how those values potentially come into
conflict with users’ values. Winner’s argument that technical artifacts
“embody specific forms of power and authority” (Winner, 1980) is suggests that
the features of a particular application can afford or disafford certain uses
in ethically relevant ways, to differing degrees for different users.

Scholars in HCI have developed various approaches—including Value-Sensitive
Design (Friedman et al., 2007), Universal Design (Hamraie, 2013), and Design
Justice (Costanza-Chock, 2020)—that draw a connection between standpoint and
affordance in design and HCI. Standpoint is the idea that people’s experiences
are shaped by their subject position—including their experiences, identity,
and relationship to power structures. Costanza-Chock argues that “design
constantly instantiates power inequality via technological affordances” in a
way that reproduces existing, intersecting structural inequalities (Costanza-
Chock, 2020). Because there is no universally normative user, designers must
contend with the fact that the same design might be received differently by
(and have different effects on) different people.

Just as theories of design can influence ethics, researchers have explored how
theories of ethics can influence design. Building from existing theories of
autonomy in feminist moral philosophy, Zong and Matias demonstrate the use of
procedural and substantive theories to analyze the ethics of software design
(Zong and Matias, 2022). Procedural theories are concerned with “standardized,
repeatable steps that can be automated by a software system or a bureaucracy …
that protect individual rights, such as a right to individual autonomy” (Zong
and Matias, 2022). When researchers and practitioners create guidelines for
ethical design, or create lists of dark patterns to avoid, these are
procedural approaches to ethics. Substantive theories are concerned with
differences in normative values between users and designers, and the use of
power in resolving those differences. When researchers and practitioners have
value-driven conversations about ethically-relevant design questions, they are
applying substantive theory. Substantive analysis requires a situated
understanding of the kinds of users who might use a system, their values and
interests, and the relevant structures that might constrain the full
expression of users’ values.

Our work contributes a framework for understanding the ethics of software
design in terms of the fit between a user’s conceptual understanding of a
system’s functionality and their expectations of that functionality. Users’
understanding of a system is shaped by the way designers communicate the
system through design choices, and designers can take procedural steps to
limit misunderstanding by drawing on users’ expectations of other applications
that behave similarly. Our concept-based framework contributes also to
substantive analysis, by grounding (and scoping) discussions about values in
established patterns of functionality.

##  3\. Background: Concepts in Software Design

In The Essence of Software (Jackson, 2021), Daniel Jackson introduced concepts
as a central unit of analysis in software design. Concepts are abstract, self-
contained units of functionality consisting of states and actions that
interact to fulfill a specific purpose. Concept design differs from existing
software design processes by encouraging designers to specify the observable
behavior of a system without needing to represent it concretely in code or a
user interface. With concepts (as with formal specification (Gaudel, 1994) and
abstract data models (Liskov and Zilles, 1974) previously), designers can
express the essence of functionality ideas prior to and independently of
representation choices. Unlike previous approaches, however, concepts allow
the functionality to be described in a modular way, aligning with the user’s
mental models and needs, and capturing similarity of function across systems.
For example, common concepts in shopping websites include Shopping Cart,
Order, Catalog, Account, Notification, etc. Below, we describe the most
relevant parts of a concept necessary to understand the rest of the paper.

Just as an application can be viewed as a state machine comprising actions on
a state, so can a concept. And an application can be viewed as a composition
of concepts that act as independent services, connected by synchronizations.

Concepts do not depend on or reference other concepts, but can nonetheless be
composed through synchronizations and by the instantiation of generic types.
For example, Shopping Cart is defined with Item as a generic type. Item may
represent clothes or concert tickets when instantiated in an application. To
articulate the functionality of a concept, we describe the concept’s states,
actions, synchronizations, and mapping:

  * •

State. A concept’s state keeps track of values that may potentially change
with user interaction. For example, a Reservation needs to maintain state
representing a pool of available resources and track which resources are
allocated to which user.

  * •

Actions. Actions characterize the behavior of a concept. Actions are used to
modify state, and state should not be modifiable except through an action.
Actions often contain preconditions that limit when the action may occur. For
example, in a Shopping Cart, a user can only remove an item from their cart if
the item is currently in the cart. In mixed-initiative systems, there may also
be constraints on who can initiate an action. For example, in a Shopping Cart,
the system should not be able to initiate adding items to the cart without
user input.

  * •

Synchronizations. A synchronization ties two actions from different concepts
together in a specific order. If action A is synchronized with action B, then
when action A is executed by a user, action B is automatically executed by the
system. For example, Facebook has a Notification concept and a Like concept.
The actions of liking a post and sending a notification to the poster are
synchronized in the app.

  * •

Mapping. A concept’s UI mapping describes how abstract functionality (states
and actions) is presented to a user as user interface elements. A mapping
draws a correspondence between each state and action to a concrete
representation in a user interface, so that a user can view state and access
actions. A good mapping results in a user interface that accurately and
transparently communicates the concept’s functionality.

We provide examples of concept definitions that include state, actions, and
synchronizations, and an example UI mapping. Figure 1A and Figure 1B show
complete concept descriptions of the Label and Todo concepts following the
structure outlined above, and Figure 1C shows a simple app design that
composes these two concepts together through a synchronization into a simple
todo app. Figure 2 shows an example of mapping the Subscription concept to a
user interface.

(A)

(B)

(C)

Figure 1. Concepts and syncs for an example todo app from (Jackson, 2021). (A)
A complete formulation of the Todo concept which allows users to maintain a
set of tasks to complete. (B) A complete formulation of the Label concept
which allows users to attach labels to items to organize them into categories.
(C) A simple app named todo-label that is composed of the Todo and Label
concepts. The delete() action from the Todo concept is synced with the clear()
action from the Label concept so that when a task is deleted from the todo
list, all of the task’s labels are removed. In a richer design presented in
(Jackson, 2021), labels corresponding to a todo being pending or done are
introduced, and competing a task is synchronized with changing the label, thus
allowing label filters to include to-do state.

Descriptions of the Todo and Label concepts as well as a todo-label app. (A)
The Todo concept’s purpose is to keep track of tasks. Its state consists of
the done and pending sets, and its actions include add(), delete(), and
complete(). (B) The Label concept’s purpose is to organize items into
overlapping categories. Its state consists of the labels set, and its actions
include affix(), detach(), find(), and clear(). (C) The todo-label app
includes the Todo and Label concepts, syncing todo.delete() with
label.clear().

Figure 2. Mapping Subscription to the Netflix user interface. Each user-
viewable component of state and user-controllable action in the concept
corresponds to an interface element.

Subscription concept definition with arrows showing mapping to the Netflix
user interface. The Subscription concept’s purpose is to manage a collection
of users and when they sign up. Its state includes subscribers, recurringCost,
renewalFrequency, startDate, and renewalDate. Its actions include subscribe(),
unsubscribe(), update() and send(). The first two are user controllable and
the latter two are subscription provider controllable. Next to the concept
definition is a screenshot of the Netflix homepage and a user account page.
There are arrows pointing from the components of the concept definition to
their corresponding elements in the user interface.

##  4\. Theorizing Dark Patterns as Concept Design

We present a theory that allows designers, scholars, and policymakers to
reason about dark patterns, and ethical considerations in interface design
more broadly, in terms of concept design.

###  4.1. Motivation: Descriptive, Evaluative, and Generative Power

Although dark pattern researchers have developed multiple taxonomies and
definitions, existing theories share important limitations. Human-computer
interaction researcher Michel Beaudouin-Lafon suggests evaluating theoretical
frameworks in terms of descriptive, evaluative, and generative power
(Beaudouin-Lafon, 2004). For theories of dark patterns, descriptive power
refers to how well a theory provides language to describe existing cases. In
other words, for a given interface, we want to be able to say whether
something is or isn’t a dark pattern. Evaluative power refers to the ability
to make comparisons between relevant features of two different cases. For
instance, given two interfaces, we want to be able to articulate what features
of each makes one more or less “dark” than the other. Finally, generative
power refers to the ability of a theory to inform new design. We want theories
of dark patterns not only to be able to tell us that there is an opportunity
to improve the design, but also to guide us in how to improve that design.

Existing theories of dark patterns are primarily descriptive. Many definitions
and taxonomies refer to specific features or design techniques that provide
language to identify when a dark pattern is present in a given interface.
Consider an example dark pattern that researchers have named “forced action”.
Mathur et al. described forced action as “Forcing the user to do something
tangential in order to complete their task” (Mathur et al., 2019), while the
UXP2 lab describes it as “any situation in which users are required to perform
a specific action to access (or continue to access) specific functionality”
(Gray, 2022). Forced enrollment is a common example of forced action—for
instance, when an online shopping website requires a user to make an account
in order to complete a task. Figure 3 shows a user interface that Mathur et
al. cite as one an example of forced action, a persistent sign-up modal that
prevents a user from using a shopping website until they have made an account.
With Mathur et. al’s descriptive theory, we are able to articulate the
particular techniques that make this design dark. Specifically, it is
asymmetric (requiring completing the additional, tangential tasks, creating
unequal choices) and restrictive (mandating enrolling in marketing
communication or creating accounts) (Mathur et al., 2019).

Figure 3. Modal that appears upon opening therealreal.com. Although it looks
like modals that commonly open on shopping websites, this modal cannot be
dismissed. The site does not clearly communicate that the implemented concepts
includes the membership concept, which results in the expected concepts not
matching the observed concepts.

Un-dismissable sign up modal on therealreal.com. The modal states ”Sign up &
get a $25 site credit” and contains a space for the user to enter their email.
Below the email field are alternative options to sign up with Apple, Facebook,
or Google. There is no button to dismiss the modal in order to access the
website behind it.

Figure 4. Misfits Market membership-only business model. Users cannot use the
website unless they create an account, which is a forced action. However,
because the business model is clearly advertised and justified on the site, it
is not clear if this design should be considered a dark pattern.

Misfits Market homepage explaining their business model in three sections. The
leftmost section is titled ”Join for FREE” and states that one save up to
$1200 every year by shopping with them. The center section is titled ”Skip the
Store” and describes the variety of groceries to choose from. The rightmost
section is titled ”Do Some Good” and states that joining the site will help
fix a broken food system and build a more sustainable future.

While descriptive power is important in helping researchers unite disparate
instances of problematic interface design under the category of “dark
pattern,” purely descriptive approaches are less helpful in evaluating
ambiguous cases. For instance, many websites have behaviors that match the
forced action definition, despite having reasonable-sounding explanations for
their functionality. Many news organizations require readers to create an
account and subscribe before reading articles, with the goal of creating a
sustainable business model for journalism. Substack, a newsletter platform,
allows writers to mark some posts as subscriber-only so that writers can offer
free articles while generating income from paid content. Some eCommerce sites
even advertise being “members-only” as part of their business model; Misfits
Market asks users to “join” to shop and “build a more sustainable future”
(Figure 4). While these sites all involve forced action and are asymmetric and
restrictive, it’s not clear that all users would find all of these examples
the same degree of frustrating or unreasonable. Taxonomy-based theories are
not able to help us comparatively evaluate whether these cases are all “dark”
to the same degree.

Taxonomies are also limited in their generative power, because they do not
provide guidance on how to improve the designs. Something like a forced action
that requires signing up for an account or a subscription may be a dark
pattern, but those design patterns are deeply intertwined with the
functionality of a website so that simply removing the forced action may not
be possible. Ideally, a generative framework could help identify solutions
that balance the designer’s goals with the user’s interests.

To make progress on evaluative and generative theories of dark patterns, we
argue that theories need to explicitly account for user expectations. We
should not understand dark patterns only in terms of features of their design,
but also in terms of how the design meets or fails to meet users’ expectations
and in terms of the values that shape how designs are received by users. In
the rest of this section, we show that by accounting for user expectations, we
can evaluate differences between designs, and generate alternative designs
that are more in line with expectations.

###  4.2. Definitions: Observed, Implemented, and Expected Concepts

To think systematically about users’ expectations, we identify three
variations of a concept that arise from users’ interactions with a software
application: the observed concept, the implemented concept, and the expected
concept. By comparing these concepts, we can identify whether a design aligns
with user expectations. If a design violates those expectations, analyzing
differences in these concepts lends further clarity to users’ perceptions of
how ethical the design is, and can suggest potential changes to re-align
expectations.

The observed concept is the concept that the user observes when they interact
with the application. Concepts include the actions available to a user, and
the state that supports those actions. The observed concept is formed via the
state and actions a user sees as available to them on the user interface. In
other words, the observed concept is the user’s interpretation of how the
application works. Regardless of the intentions of the designer or programmer,
the observed concept represents the user’s perceived experience of the
application.

The implemented concept is the concept that is implemented in the actual
functionality of an application. The implemented concept is programmed into
the application, establishing which actions are possible to take and how those
actions impact the state. It does not depend on the user at all; the
functionality is inherent in the application itself. When a concept is mapped
well to a user interface, the observed concept will closely match the
implemented concept; all the actions will be visible to the user and the user
will have a clear and accurate understanding of the state. However, when a
user interface does not adequately communicate the implemented concept, the
observed concept will differ more drastically due to gaps in the user’s
understanding. For example, a subscription service may have an implemented
concept that enables both subscribing and unsubscribing. But if the UI mapping
makes unsubscribing very difficult so that the user does not feel they can
access the action, the user’s observed concept will not include an unsubscribe
action. In cases where an application contains multiple concepts, a UI could
potentially not only fail to communicate certain states and actions, but fail
to communicate the presence of entire concepts.

The expected concept is the concept that a given user expects to be
interacting with, based on their experiences with that concept in the past.
One of the key principles of concept design is that concepts are reusable
components of an application; users are familiar with how concepts should
function (their actions, states, synchronizations, etc.), due to having
interacted with the concept across many applications. Common concepts, like
Account or Shopping Cart, appear on enough websites that many users have well-
formed expectations for how they work. The user interface of an application
communicates to the user what concepts are present, typically through using
common interface elements that are frequently used for a given concept. Based
on what concepts are present, the user then applies their existing
expectations for the functionality of the various concepts. Due to the reuse
of concepts, most users will have a very similar expected concept; indeed,
designers make applications more intuitive to users by reusing existing
concepts (Jackson, 2021). However, the expected concept is still subjective
and can vary across different users, particularly for concepts that are newer
or quickly evolving. Identifying the variations in the expected concept across
different user groups can be critical to understanding how different users
will react to an application.

Above, we introduced the notions of observed, implemented, and expected
concepts within the context of a single concept. It’s important to note that
these categories also apply to compositions of concepts, both in how concepts
are composed (that is, which synchronizations are present) and even which
concepts are included. For example, EmailMessage and Label are frequently used
in conjunction with each other, so users have developed expectations for their
synchronizations. For example, when an email is sent, a user may expect that
the “sent” label is added to it. In a given design, it is helpful to both
analyze individual concepts and small groups of concepts that together provide
key functionality to the system.

###  4.3. Procedural and Substantive Analysis of Dark Patterns Using Concepts

What it means for an interface to be “ethical” is shaped by the theories of
ethics that designers and researchers bring to their analyses. Zong and Matias
identify two important classes of ethics theories: procedural and substantive
theories. Procedural theories are concerned with “standardized, repeatable
steps that can be automated by a software system or a bureaucracy … that
protect individual rights, such as a right to individual autonomy” (Zong and
Matias, 2022). Arguments that interfaces are ethical if they don’t include
certain kinds of design patterns, or if designers follow certain steps to
enable individual choices, are procedural arguments that are intended to apply
across scales and contexts. In contrast, substantive theories are subjective
and context-specific, concerned with differences in normative values between
users and designers. Substantive analysis requires a situated understanding of
the kinds of users who might use a system, their values and interests, and how
designers exercise power in resolving (or ignoring) differences between users’
values and the values expressed by the system. Zong and Matias argue that
procedural and substantive theories are two complementary perspectives that
“provide resources for combining equally-important considerations” (Zong and
Matias, 2022). As such, we find that both perspectives are essential to a
complete understanding of dark patterns.

We introduce a two-part definition that integrates both procedural and
substantive considerations when analyzing dark patterns using concepts.

A design is dark for a given user when:

  1. (1)

The observed or implemented concept is different from the user’s expected
concept;

  2. (2)

The difference benefits the designer or owner of the system to the detriment
of the user.

Consider the hypothetical case of two friends out to eat at a restaurant, one
American and one European. At the beginning of their meal, they ask the waiter
for water for the table. When they receive the check, they find a line item
charging them for the water. To the American, this feels outrageous and
deceptive, an extra charge being snuck onto the bill so that the restaurant
could make more money. But to the European, who is accustomed to paying for
water, it is nothing of note. The behavior of the restaurant was the same, but
the expectations of the two customers varied, resulting in this feeling like a
dark pattern to one but not the other.

####  4.3.1. Procedural Considerations

When a design breaks a user’s expectations of its behavior, because their
expected concept did not match either the observed or implemented concept, it
negatively affects the user’s perception of the ethics of the system. In the
restaurant, both the American and European had strong expectations of how
billing at restaurants should work, based on their prior experiences
interacting with restaurants. When reality broke from these expectations for
the American, it was jarring. Similarly, users have accumulated expectations
about how common concepts should function, and it feels jarring when these
expectations are not met. In contrast to ordering water at restaurants, the
global nature of the internet means that users across the world interact with
the same concepts online. While different users may have different nuances to
their expectations based on their backgrounds and experiences, they still
build shared, core expectations for how concepts should function.

Observed concept violates expected concept: When the observed concept breaks a
user’s expectations of an existing concept’s behavior, the user interface is
communicating functionality that violates the user’s preconceptions of how the
application will behave. This violation could be due to an action not being
available to the user, a state not representing the expected state, or other
changes to the concept’s functionality.

Example: Brignull’s “hard to cancel” dark pattern occurs when “the user finds
it easy to sign up or subscribe, but when they want to cancel they find it
very hard” (Brignull et al., 2023). In this case, Subscription is the relevant
concept. The user’s expected concept is that Subscription has both a subscribe
and unsubscribe action. However, when the user interacts with the application,
their observed concept is that unsubscribe is not available. Even if the
implemented concept includes an unsubscribe action (such that it is
technically available), if the user finds it very difficult to access, their
observed concept will not include the action. Therefore, the observed concept
violates the expected concept.

Implemented concept violates expected concept: In some cases, the observed
concept, as projected by the UI and the interactions the user has with it,
appears to conform to the concept the user expects, but the underlying
behavior is not as expected. In this case, the expected concept matches the
observed concept, but not the implemented concept. The design may not raise
any flags for the user at the moment of interaction, but the implementation
does not match their expectations. This mismatch may become clear to the user
in the future (ex: placing an online order but the item never arrives), may
only be clear to some users (ex: a bug that causes certain items not to be
added correctly to a shopping cart), or remain hidden indefinitely (ex: a
recommendation system claiming to rank content by relevance, but actually
showing sponsored content).

Example: Brignull’s examples of fake scarcity, fake social proof, and fake
urgency are examples of the implemented concept violating the expected concept
(Brignull et al., 2023). For example, in fake scarcity, the expected concept
is that the state of the concept (ex: an Inventory’s count of items) is
accurately displayed. However, in fake scarcity, the implemented concept does
not behave as expected. Instead of displaying the count of items in the
Inventory, the UI displays an arbitrary number. However, due to the mapping of
the UI, the fact that the display does not actually represent the concept’s
state may not be apparent to the user. In this case, the observed concept
fulfills the expected concept, but the implemented concept violates the
expected concept.

####  4.3.2. Substantive Considerations

Deviating from an expected concept isn’t inherently unethical—it matters what
outcome is produced by that deviation, and whether users are harmed by that
outcome. Consider the restaurant scenario again, but the pair is not charged
for the water. In that situation, the European’s expectations are broken, but
the outcome is a welcome surprise, and not experienced as dark. The core
difference is that when the American is unexpectedly charged for water, that
unexpected charge felt like the restaurant taking advantage of them. When the
European is unexpectedly not charged, they feel like they unexpectedly
benefited. Procedurally, these two cases were the same in that someone’s
expectations were violated. But substantively, the way that violation
interacts with the person’s values impacts how they perceive the interaction.

While both procedural and substantive perspectives are essential to a complete
understanding, prior work in dark patterns has typically been purely
procedural. While Mathur et al. do call for more normative approaches to dark
patterns (which would be considered substantive), their recommended normative
considerations tend to focus on a limited set of liberal values (for example,
autonomy and fairness in markets) (Mathur et al., 2021). As data ethics
scholars have noted, there are important dimensions of justice that are not
included within a rubric of rights, opportunities, and material resources
(Hoffmann, 2019). Instead of recommending universalizable normative concerns,
we instead recommend a situated, context-specific approach centered in users’
subjective expectations and values.

Users have subjective values. Values shape how users perceive the outcomes of
their interactions with interfaces. By values, we mean ideas that shape what
users believe to be good, preferable, or acceptable. Consider a user who is
choosing between different shopping services to use. If they highly value
efficiency, they may pick the service with the fastest shipping times. If it
is important to them to minimize their spending, they might pick the cheapest.
If they value environmental sustainability or workers’ rights, they might pick
one that is slower or more expensive, but has a lower carbon footprint. When a
system acts against a user’s values, this can adversely affect their
experience and perception.

In a specific situation, how do designers know what these goals, beliefs, and
values actually are? The only way to really know for sure is to talk to users.
That’s why having communication channels for feedback and accountability is
really important for designers who want to understand the substantive
landscape they are working within. But as Zong and Matias note, directly
asking users about their values up front is not always practical or possible
(Zong and Matias, 2022). For instance, users of secure, anonymous
communication apps would view outreach from designers as a severe privacy
violation. Absent explicit knowledge about a user’s values, designers need to
make assumptions on their behalf. It is crucial for designers to critically
reflect on these assumptions, starting with the realization that assumptions
are always present and it’s better to make them explicitly with justification
than to leave them unexamined. In other words, designers should have an
explicit model of how they expect users to perceive the ethics of their
decisions, and to be able to justify the assumptions reflected in that model.

Users’ values may conflict with the designer’s values. In many of the
conventional dark pattern examples, deceptive or misleading techniques are
employed to steer users toward outcomes that prioritize the designer or system
owner’s values, rather than the user’s. For instance, Brignull’s “hard to
cancel” pattern makes it difficult for users to cancel recurring subscriptions
(Brignull et al., 2023). In this case, the company is attempting to maximize
their revenue, which often comes into conflict with the user’s immediate goal
to cancel their subscription and keep their money.

Different users’ values may not be simultaneously satisfiable. Duolingo’s use
of frequent notifications to promote daily usage highlights how the same
design can feel acceptable or unacceptable to different users. One user
expressed their frustration at “emotionally manipulative notifications about
losing streaks I didn’t want to have” (u/frankie_co_su, 2020). But another
user felt that these tactics were ultimately for their own benefit, saying
that “Duolingo uses a ton of predatory/addictive dark ux to keep people
engaged with their app, but the thing they’re engaging with is learning a new
language, so it’s actually fine” (cohost.org/hthrflwrs [@HTHRFLWRS], 2023).
Both users hold reasonable normative positions; one weighs the value of being
compelled to learn a language higher than the adverse effects of that
compulsory messaging, and the other simply does not. Duolingo’s designers are
choosing to prioritize the values of the first set of users over the values of
the second.

Individual autonomy may conflict with the common good. While it is important
to consider individual users’ values, only doing so may reproduce structural
inequality and discrimination on a societal level. When a dating app shows a
user other people’s profiles that align with the user’s subjective
preferences, for example, the app may be aligned with the user’s values. But a
designer might bring the consideration that society would be well served by
discouraging preferences that lead to greater separation by race, ethnicity,
religion, gender, etc. In this case, a top-down intervention of this sort
would likely not be well received by users who might find it paternalistic and
invasive to have an algorithm override their personal preferences. In general,
designers must decide how to balance balance the values of individual autonomy
and the common good.

Concepts ground substantive analysis in concrete functionality. Substantive
analysis can feel difficult to do because it is context-specific, involves
subjectivity, and requires thinking both individually and collectively.
Concepts can help scope substantive evaluations by using concrete aspects of
an application’s functionality as starting points to discuss ethical outcomes.

Consider the case where the observed or implemented concept does not match the
user’s expected concept. Identifying this mismatch allows designers to then
narrow in on the specific part of the concept (an action, state,
synchronization, or mapping) that deviates from expectations. Then, they can
think counterfactually about how that deviation creates outcomes that may not
have happened had the expected concept been correctly implemented or
communicated.

Concepts provide language for discussion specific choices made when designing
an application’s functionality or its interface. These choices contribute to
the overall set of normative values that designers embed into their designers.
Concepts provide an additional tool for designers to scaffold conversations
around articulating and justifying these choices.

##  5\. Concept Catalogs: Articulating Standard Expectations

Thinking with concepts allows us to identify common expectations that arise
from frequently reused concepts. Typically, however, concepts and their
associated expectations are not documented. To enable designers and
policymakers to articulate concepts that can be referenced, implemented, and
critiqued, there is a need to define standard concepts that reflect
commonplace expectations. In this section, we introduce concept catalogs, a
tool for formally defining and communicating standard expectations for
concepts, and discuss how designers can map a standard concept to a user
interface. We then demonstrate how to create concept catalog entries entries
via an example case.

###  5.1. The Need to Standardize Expected Concepts

####  5.1.1. Why is standardization necessary?

Our framework for dark patterns requires a clear understanding of user
expectations for a given concept; however, these expectations are not
currently documented or standardized. Designers who are experts in a given
type of application may have an intuition of common user expectations, but
they do not have the means to share this knowledge. Additionally, while most
users have similar expectations for a given concept, their exact expectations
can vary due to evolving concepts (e.g., whether posts should have the option
to be shown chronologically) or not reflect their best interests (e.g.
expecting poor privacy protections). Standards are needed to enable
stakeholders to clearly articulate the expectations they intend to abide by or
enforce compliance to, which will enable more precise analysis of when those
expectations are being violated.

####  5.1.2. What is a concept catalog, and why is it useful?

Concept catalogs are a means to create these standards, by creating a
collection of concept entries that define standard expectations for each
concept. Concept catalogs were first proposed in The Essence of Software
(Jackson, 2021), with the idea that they would “[allow] designers to codify
their knowledge, making it easier for novices to acquire expertise, and would
encourage greater reuse of concepts and help designers avoid known pitfalls”
(Jackson, 2021). While concept catalogs are a potentially useful tool for
design more generally, they also can be particularly powerful in analyzing
dark patterns by enabling all stakeholders to understand user expectations of
a concept. Building the catalog from concepts with a focus on functionality
allows for more precise design analysis, in comparison to existing tools such
as platform design guidelines (e.g. Google Material Design (noa, 2023d) and
Apple Human Interface Guidelines (noa, 2023c)) or design pattern catalogs
(e.g. ui-patterns.com (noa, 2023g), GoodUX (noa, [n. d.]), UXarchive (noa,
2023h)). These tools tend to be too specific, being limited to a particular
platform or UI framework, or too broad, focusing on the generality of UI
widgets without considering the functionality behind them.

Even without consensus on what the standard expectations should be for a
certain concept, a concept catalog entry allows designers to explicitly
articulate a point of reference that they are designing toward. The systematic
and reusable nature of concepts allows the defined expectations to be both
precise and applicable across domains. Standard concepts can provide the
language to ground critiques during design discussions and give designers a
baseline to work from. They can also help companies meet a consistent set of
expectations across their products, and help regulators communicate and
enforce expectations that are particularly critical to protecting users.

###  5.2. Creating Concept Catalog Entries to Define Standard Concepts

A standard concept defines a set of standardized expectations that all
implementations of the concept should fulfill. A concept catalog is a
collection of entries, each defining a standard concept. In this section, we
define the structure of a concept catalog entry, provide an example entry, and
show how standard concepts can be incorporated into our concept analysis
framework.

####  5.2.1. Components of a Concept Catalog Entry

A concept catalog entry contains the concept definition and the concept
mapping. Together, they specify the standard core functionality of a concept
and its standard mapping to a user interface.

Concept Definition. The concept definition should enumerate the concept’s
state and actions. It should also define expected synchronizations—or, when an
action in one concept results in an action being called in a different
concept.

For mixed-initiative systems, the definition should specify who is allowed to
initiate each action. For example, in a Shopping Cart, it’s expected that only
the user should be allowed to initiate the action that adds items to the cart.
If the system were to initiate this without the user’s involvement, that would
typically violate expectations.

Different implementations of the same concept may include various extensions
on the concept’s core functionality. A standard concept is meant to capture
the minimal functionality that is essential to forming users’ expectations.
Almost all applications will implement extensions beyond this minimal
definition, but these extended implementations must not conflict with the
expectations established in the minimal standard concept. Scoping standard
concepts to only include core functionality enables designers to use the
catalog while maintaining the flexibility to innovate and extend their
applications.

The concept definition can be articulated in formal concept language (as in
Figure 5) or in plain language. While defining a standard concept in plain
language is often more useful to designers using the entry due to readability,
defining a formal concept requires additional precision, which can be helpful
for improving the quality and completeness of the concept.

Concept Mapping. For each action and state within the concept definition, the
catalog entry can also specify standards for mapping the concept to a user
interface. A mapping exposes a concept’s actions and state as user interface
components, like buttons and text labels. Just as users form expectations
about functionality, they also expect mappings to follow patterns they have
seen in past experiences with software applications. If a user encounters an
unexpected mapping, their observed concept may be inaccurate or reflect a
different concept entirely. To prevent this, catalog entries can specify
standard mappings for a given concept that designers can use.

For each state and action in a concept, the catalog entry should describe any
key mapping requirements. While there are general usability principles that
apply to mapping, which we discuss in 5.3, there may be mapping requirements
that are specific to a concept or a context of use. Mapping standards only
need to describe the minimum requirements for the UI to convey the concept’s
functionality to a user—they do not include implementation details or
stylistic guidance.

####  5.2.2. Shopping Cart: An Example Catalog Entry

In this section, we define an example Shopping Cart standard concept and
provide example mapping standards. We show the essential parts of the formal
definition in Figure 5. A full version can be found in Appendix A.

Concept Definition The concept’s states, actions, and synchronizations are
written to be reusable (applicable to any shopping cart) and minimal (only the
required, core parts of the concept are defined).

States. The Shopping Cart concept has state linking each cart to an owner and
a set of items. Each item has a quantity and price, from which the subtotal is
derived.

Actions. The Shopping Cart concept defines actions required to update its
state, along with whether the user or shop can initiate each action. These
actions include initializing the cart, adding and removing items, updating
item quantities and prices, and checking out.

Synchronizations. The Shopping Cart is synchronized with Catalog (to ensure
matching prices and accurate stock counts) and Order (to create an order when
a shopping cart is checked out). Defining standard synchronizations help the
implementation match users’ expectations of functionality that spans multiple
concepts.

Concept Mapping. The mapping standards for subtotal and quantityChange are
examples of state and action mapping standards. These standards provide
positive guidance, describing how the UI should behave rather than listing UI
elements to avoid. Additionally, they focus on functionality instead of
prescribing implementation details such as components, style, or copy-writing.

(A)

(B)

(C)

Figure 5. Shopping cart concept catalog entry that designers can can compare
to check if their design fulfills the requirements set by the standard
concept. (A) Standard, formal concept definition, articulating the purpose,
state, and actions. (B) The required synchronizations for the shopping cart,
which helps designers meet user expectations of functionality between
concepts, in addition to within individual concepts. (C) Example standards for
how to map the state (ex: subtotal) and actions (ex: quantity change) to the
user interface.

All components of the figure are described in the text of section 5.2.2.

####  5.2.3. How To Make and Use a Catalog Entry

Defining the concept definition and concept mapping requires an understanding
of how the concept is implemented by existing applications. Concept design
theory allows us to group expectations by concept; rather than trying to
define the expectations for entire applications or types of application, we
can break the designs down into reusable concepts. While different
applications may extend concepts in different ways, the core functionality of
the concept will be very similar across applications. Due to the frequent
reuse of concepts, users form expectations for this core functionality that
they come into contact with across many applications. By surveying existing
applications to identify this core functionality and how it is typically
mapped to the user interface, a stakeholder is able to write a complete
standard concept.

When we have access to a well-defined standard concept, we can incorporate
that standard concept into our concept analysis. Rather than analyzing an
interface in terms of an individual user’s expectations, which can potentially
differ from other users’ expectations, we can re-formulate our previous dark
pattern definition using a standard concept. Because standard concepts
incorporate some notion of common expectations, removing the need to consider
subjective differences in expectations, this makes it easier to think reason
in the aggregate about likely outcomes for most users.

For a standardized concept, a dark pattern occurs when:

  1. (1)

The observed or implemented concept does not include the standard concept’s
functionality

  2. (2)

The difference from the standard concept benefits the designer or owner of the
system to the user’s detriment

As with our previous definition using expected concept, using standard concept
allows us to think in terms of where an application might diverge from
standard expectations of functionality. While different standards may emerge
for the same concept, organizations can develop catalogs to articulate the
standards they intend to follow. Designers and researchers can use the
standard concept definition appropriate to their context. Regulators and
organizations can also compare and evaluate different ways of defining
standard concepts, advocating for the adoption of definitions that have been
thoroughly reviewed.

###  5.3. Mapping Principles

Concept catalog entries include concept-specific standard mapping guidance,
but there are also general mapping principles that apply to all concepts.
Here, we discuss a few relevant usability principles that help designers
clearly communicate a concept’s state and actions.

Transparency. The UI should accurately and fully convey all parts of the
concept that are available to the user.

  * •

There should be a 1:1 mapping between the standard concept’s user-viewable
states and user-controllable actions, and visible user interface elements. In
other words, each state and action should correspond to visible and accessible
UI elements, and there should be no elements that imply states or actions that
are not part of the concept.

  * •

The UI should truthfully convey the value of each state component, and
accurately convey what actions are triggered by each interactive element.

Usability. The UI should follow conventional usability principles (Norman,
1998; Nielson, 2020; Tognazzini, 2014). Here, we highlight a few important
principles for mapping:

  * •

Consistency: When the same state or action is represented in multiple places,
those representations should all look and behave the same.

  * •

Symmetry: Actions that represent alternative options for a choice (e.g.,
accept and decline) should be given equal prominence in the interface.

  * •

Conventions: Some states and actions have conventional representations that
are well established. For example, an “x” typically represents cancel or exit.
If these representations are used, they should represent the expected state or
action. (Note that this is arguably another kind of consistency, albeit across
applications.)

##  6\. Evaluation

In this section, we evaluate the descriptive, evaluative, and generative power
of our framework. We evaluate descriptive power by demonstrating that our
concept-based framework can cover Brignull’s existing taxonomy of dark
patterns (Brignull et al., 2023). Where the taxonomy treats each pattern as
equally dark, our concept is able to descriptively identify different reasons
why each pattern is dark—leading to a more nuanced understanding of each
pattern’s complexity. We also evaluate the descriptive power of our Shopping
Cart concept example, showing that the standard concept is able to describe
the core functionality of 20 top e-commerce sites. We show the framework’s
evaluative and generative power through a case study involving three seemingly
similar e-commerce websites. Using our framework, we are able to compare the
three cases to understand why some feel comparatively more dark to users than
the others, despite containing the same design patterns. We are also able to
suggest concrete changes to the cases that would make them feel less dark.

###  6.1. Descriptive Power

####  6.1.1. Coverage of Existing Dark Pattern Taxonomies

To evaluate the ability of the introduced theory to describe existing dark
patterns, we re-describe each dark pattern in Brignull’s Deceptive Patterns
taxonomy (Brignull et al., 2023). We selected this taxonomy because it was
recently updated in April 2023 to include major dark patterns documented in
the literature, including all patterns identified by Gray et al. (Gray et al.,
2018), Mathur et al. (Mathur et al., 2021), Bosch et al. (Bösch et al., 2016),
and by Brignull’s original list (Brignull, 2010). Table 1 identifies the
concepts most commonly associated with each of Brignull’s deceptive designs
and briefly explains how our framework characterizes the dark patterns in
terms of a violation of the expected concept.

In addition to describing each dark pattern in Brignull’s taxonomy, our
framework also introduces additional descriptive power. Where Brignull’s list
gives equal weight to each entry in the taxonomy, using concept analysis helps
us differentiate between various reasons a pattern could be considered dark.

For instance, some dark patterns are caused by explicit misrepresentation when
the underlying concept is mapped to a user interface. These cases include
comparison prevention, disguised ads, fake scarcity, fake social proof, fake
urgency, hard to cancel, and hidden costs. The implemented concept will always
be different from the expected concept, as user expectations are formed by a
deceptive user interface.

While these cases are egregiously dark, other dark patterns rely more strongly
on user expectations; in these cases, concept theory is able to describe
nuances that are not captured by the taxonomy. Sometimes a dark pattern
definition implicitly assumes that user expectations are universal across
users. For example, the nagging pattern reflects the expectation that a
disable action should be available whenever there are repeated requests to the
user. Similarly, the obstruction pattern reflects an implicit expectation that
some important actions (like rejecting privacy-unfriendly settings) should
always be available to the user. In other cases, whether or not something
feels like a dark pattern will depend on a user’s subjective experience. These
cases include forced action, hidden subscription, preselection, visual
interference, sneaking, and confirmshaming. For example, consider an interface
for making a donation. Most users do not expect the option to subscribe to a
recurring donation to be pre-selected, making the design feel dark. However,
if a user is donating with the intent of setting up recurring donations and
expects that to be the common use case, then the pre-selection will not
violate their expectations. These kinds of dark patterns result when the
observed concept does not match the expected concept. However, these
expectations will be highly context-specific.

Dark Pattern |  Source Example |  Concepts |  Relevant property |  Reason for violated expectations
---|---|---|---|---
Comparison Prevention |  T-Mobile: bundling prices in different ways |  Catalog |  State: price |  Mapping: usability (consistency)
Confirmshaming |  MyMedic: “No, I don’t want to stay alive” |  Notification |  Action: opt in/out |  Observed mapping of two options does not match the expectation of symmetry
Disguised Ads |  Softpedia: advertisements mimic download buttons |  Advertisement |  – |  Mapping: usability (conventions)
Fake Scarcity |  Shopify: shows fake low stock messages |  Inventory |  State: stock |  Mapping: transparency
Fake Social Proof |  Beeketing: shows fake social proof (9 customers just bought this item) |  Orders |  State: count |  Mapping: transparency
Fake Urgency |  Shopify Hurrify: creates countdown timers that reset upon hitting zero |  Countdown Timer, Catalog |  State: CountdownTimer.remainingTime, Catalog.salePrice |  Expected synchronization between timer and sale is not implemented
Forced Action |  LinkedIn: required giving access to all email contacts to use site |  Account, Contact |  Synchronization: (Account.create, Contact.share) |  Implemented synchronization does not match expectation that consenting to sharing is independent from account creation
Hard to Cancel |  New York Times makes it easy to sign up but difficult to cancel |  Subscription |  Action: subscribe/unsubscribe |  Mapping: usability
Hidden Costs |  Stubhub: advertises low price, goes through many steps, then reveals higher price |  Shopping Cart, Catalog, Order |  State: price |  Mapping: transparency, usability (consistency)
Hidden Subscription |  Figma: when user shares file with editing permissions, a new subscription is charged |  AccessControl, Subscription |  Synchronization (Subscription.subscribe, AccessControl.share) |  Implemented synchronization does not match expectation that sharing and subscriptions are independent
Nagging |  Instagram: aggressively requested users turn on notification, no option to reject fully |  Notification |  Action: enable |  Expected disable action is not implemented
Obstruction |  Facebook: makes it difficult and confusing to reject privacy-invading settings |  PrivacySetting |  Action: accept/reject |  A user who can’t navigate the menu observes unexpected lack of reject action
Preselection |  Trump campaign: preselected “make this a monthly recurring donation” |  Subscription |  Action: donate, subscribe |  A user who does not notice the box observes unexpected synchronization between donate and subscribe
Sneaking |  SportsDirect: sneaks item into the user’s shopping cart |  Shopping Cart |  Action: add |  Implemented add action is unexpectedly performed by system instead of user
Trick Wording |  Ryanair: used language to make users accidentally book travel insurance |  Shopping Cart |  Action: add |  A user who does not notice they are adding something observes add action unexpectedly performed by system
Visual Interference |  Tesla: used low contrast text to hide refund policy |  Order |  Action: refund |  A user who does not find the policy observes unexpected lack of refund action
Table 1. Brignull’s Deceptive Designs (April 2023 version) (Brignull et al.,
2023) under our framework. For each deceptive design, we provide an example
implementation, list the concepts and their specific state/actions most
commonly associated with the design, and explain the way in which the relevant
concept component violates user expectations.

####  6.1.2. Concept Catalog Coverage of Popular Websites

To evaluate whether standard concepts can effectively describe common
functionality, we conducted a review of top e-commerce sites to identify
whether the Shopping Cart catalog entry accurately describes functionality
across sites. We manually examined the top 20 e-commerce websites in the USA
as of June 2023, ranked by web traffic and engagement on Similarweb (noa,
2023f). Similarweb is an industry-standard web traffic analytics platform
(Jansen et al., 2022) that reportedly has data on over 100 million websites
(noa, 2021). We excluded e-commerce websites that did not contain the shopping
cart concept (e.g. Rakuten), as well as websites whose primary language was
not English. Table 2 shows the top 20 remaining websites.

We found that all 20 sites conformed to the core Shopping Cart concept as
described by the concept definition and mapping standards in our catalog
entry. This finding supports the use of concepts as a lens for studying
software design, since many websites re-use identical, well-understood units
of functionality. Additionally, our findings support that constructing a
concept catalog is feasible, since it is possible to reduce the core
functionality of a large corpus of websites to a concise and accurate
description.

In this review, we identified that most websites will implement extensions
beyond the minimal functionality in the standard concept. We document these
extensions in table 3. A concept can be extended in an infinite number of
ways, and it is impossible to list every possibility in a concept catalog (as
discussed in 5.2.1). Thus, a concept catalog entry only contains the minimal
functionality required for a given concept. The first part of our dark pattern
definition prompts designers to create extensions that do not conflict with
the core standard concept. In cases where extensions do conflict, the second
part of the definition helps designers differentiate between dark patterns and
harmless innovations.

Website |  Standard Concept |  Extensions
---|---|---
amazon.com |  ✓ |  1,2,3,4
samsung.com |  ✓ |  2,4,5
ebay.com |  ✓ |  2,4
etsy.com |  ✓ |  1,2
aliexpress.com |  ✓ |  1,3,4,6
walmart.com |  ✓ |  1,2,5,9
apple.com |  ✓ |  5
shein.com |  ✓ |  2,3,8,9
temu.com |  ✓ |  1,3
homedepot.com |  ✓ |  1,2,5,7,9,10
flipkart.com |  ✓ |  2,4
trendyol.com |  ✓ |  3
target.com |  ✓ |  1,2,9
nike.com |  ✓ |  -
hm.com |  ✓ |  -
t-mobile.com |  ✓ |  2
lowes.com |  ✓ |  1,2
att.com |  ✓ |  1,2,7,10
zara.com |  ✓ |  -
wayfair.com |  ✓ |  1,2
Table 2. Shopping cart concept coverage of popular websites. All of the top 20 most popular e-commerce websites in the US implement our standard shopping cart concept definition, although most websites implement extensions beyond the core functionality (Table 3). According to our dark patterns definition, websites that only contain the standard concept are acceptable. However, any extensions can also be evaluated for darkness using the substantive portion of our definition. # |  Extension |  Description
---|---|---
1 |  Multiple quantity add |  add() is augmented with a quantity parameter to allow adding more than one of an item to the cart at a time
2 |  Save for later |  Include state saved to maintain items that the user wants to checkout in a later order. Users can move items freely between items and saved.
3 |  Partial checkout |  Include state selectedForCheckout to maintain items that the user wants to checkout now. Users can select/deselect items in items to be added to/removed from selectedForCheckout, and during checkout() only items in selectedForCheckout are added to the order.
4 |  Buy now |  Include action buyNow() that is synced with fulfill() in order, allowing users to checkout a single item immediately.
5 |  Estimated total |  Include state estimatedCheckoutTotal to maintain the estimated additional shipping fees and taxes that will be added to subtotal during the checkout process.
6 |  Make account to add to cart |  Compose cart with an authorization concept such that users can only access cart actions like add() and checkout() if they have made an account.
7 |  Share cart |  Include state viewers, and compose cart with an authorization concept so that users in viewers can view (but not edit) the contents of items. Only the owner can add other users to viewers.
8 |  Cost until free shipping bar |  The UI displays freeShipping - subtotal. The state freeShipping is from the order concept containing the minimum subtotal the cart must have to qualify for free shipping.
9 |  Quick add |  The UI contains a normal add button on the product information page and a quick add button on the catalog display page where users can add an item without opening its full product information. Both buttons use the add() action under the hood.
10 |  Empty cart |  The UI contains a checkout button and an empty cart button. Both buttons use the checkout() action under the hood.
Table 3. Common extensions to the basic shopping cart concept, found in the
list of top 20 e-commerce websites (Table 2)

###  6.2. Evaluative and Generative Power: Shopping Cart Case Study

In this section, we demonstrate the evaluative power of our framework by
comparing three applications that are superficially similar, but that users do
not perceive as equally unethical. Further, we demonstrate the framework’s
generative power by showing how designers can use the specific divergence from
the expected concept to identify where they need to realign expectations.
Here, we introduce three cases of applications using similar design patterns,
which we then discuss and evaluate throughout the rest of the section.

Case 1 — Sports Direct: Brignull uses Sports Direct, an online sportswear
shop, as an example of a dark pattern (Brignull, 2013). On Sports Direct,
after a user adds items to their shopping cart, when they go to checkout they
will find that a magazine costing 1 dollar was added to their cart without
their knowledge. To remove it, they must leave the order screen and go back to
their shopping cart. This is an unambiguous example of the sneaking dark
pattern.

Case 2 — Grove Collaborative Grove Collaborative (grove.co) is an online
household shopping website that includes features that some users find sneaky
and problematic, while others find them helpful. User dissatisfaction does not
appear to be intentional—Grove Collaborative seeks to be seen in a positive
light, pitching itself as “a leading socially responsible company” (noa,
2023i). One of Grove Collaborative’s services is adding suggested items to the
cart for the user. For example. when a user opens the site, there are 13
suggested items already in their cart—some free and some that cost money (noa,
2023b). Another example is that when a user checks out, they are automatically
enrolled in a recurring subscription service, where suggested items are placed
in their cart and automatically checked out each month, described by Grove
Collaborative as “personalized, scheduled Grove shipments, so you’ll never run
out of the products you love” (noa, 2023i).

User reviews highlight the fact that some users find Grove Collaborative’s
features to be sneaky, while others find the same features to be helpful and
transparent. One disgruntled user lamented that they “had to always watch
[their] cart for extras they sneak in there” (noa, 2023b). Another felt that
it demanded extra vigilance on their end, writing, “I don’t like the add ons.
Sneaky. You have to watch your cart” (noa, 2023b). However, other reviews are
much more positive. One user wrote, “I actually like the auto-ship. I get an
email and app notification a week before my ship date, reminding me to make
sure my cart has what I need” (noa, 2023b). Users described auto-ship as
“incredibly helpful” and “very convenient” (noa, 2023b). These varied reviews
show that Grove Collaborative is not a clear-cut case of a dark pattern—user
perceptions of the system vary widely.

Case 3 — StitchFix: StitchFix is an online clothing company with similar
functionality to Grove Collective, but that users do not perceive as dark.
StitchFix puts clothing items in the user’s bag based on a style quiz they
fill out; when the user checks out, the site charges the user a fee and ships
all the items, then the user returns the items they don’t want (noa, 2023a).
By default, the user is enrolled in a monthly subscription to automatically
receive new sets of clothes. The StitchFix mode of shopping bears significant
resemblance to the Grove.co mode: suggested items are placed in the user’s
cart and shipped each month, providing an automated shopping experience.
However, users do not find that items automatically appearing in the cart and
shipping feels sneaky. Negative reviews mention poor customer service,
difficulty canceling, issues with payment refunds, and low-quality clothing.
However, they do not reflect negative perceptions of the shopping features
(noa, 2023e). The feeling of “sneakiness” is not mentioned in any of the
approximately 200 reviews (noa, 2023e).

####  6.2.1. Dark Pattern Taxonomies are Insufficient

Although these three websites elicited very different responses from users,
existing taxonomies of dark patterns cannot fully account for these
differences. Sports Direct is well described by Brignull’s sneaking dark
pattern, which is defined as a case where “the user is drawn into a
transaction on false pretenses, because pertinent information is hidden or
delayed from being presented to them” (Brignull et al., 2023). But even though
quite a few users found Grove Collaborative to be dark, it does not clearly
meet this definition. The company markets auto-ship as a valuable part of the
service, not something that is snuck behind the user’s back. The user
interface clearly communicates that items are in the cart and allows the user
to remove them, as shown in Figure 6. Before a recurring order is shipped,
reminders are sent via text and email both a week before and the day before,
giving users the option to cancel or make changes. But despite this
transparency, items being automatically added to the cart by the store
intuitively feels dark to some users as evidenced by negative reviews. Yet,
the existence of positive reviews shows that not all people perceive Grove
Collective as a dark pattern. Further, Stitchfix has almost identical
functionality to Grove Collective, yet does not feel dark to most users.
Through the lens of a taxonomy, design techniques either do or do not match a
dark pattern definition. A taxonomy does not provide language to discuss why a
design may feel subjectively dark to some users but not others.

Figure 6. The Grove Collaborative user interface clearly communicates that
items have been added to the cart (through both text and icons) and that they
can be removed. Despite this clear communication, however, some users still
feel that the service is ”sneaky”.

Banner in the Grove checkout user interface. Underneath the title ”Review your
order”, the site explains that they’ve added free gifts to the cart which can
be removed be the user. The cart icon next to the text indicates that 13 free
items have been added to the cart.

####  6.2.2. Concept Analysis: Evaluating Cases and Generating Solutions

By using concepts to analyze the ambiguous cases in both procedural and
substantive terms, we can understand where important differences arise.

In the Grove Collective case, the observed concept does not match the expected
concept. Grove Collective’s design presents a Shopping Cart concept to the
user via conventional shopping cart mappings (shopping cart icon, add to cart
buttons, quantity increase / decrease icons, etc.). Because of this, users
expect to be interacting with shopping carts similar to ones they have used in
the past. Most users expect that only the user can initiate the add() action
in a shopping cart; the shop calling add() without their involvement breaks
this expectation. In the sneaking dark pattern, the site tries to hide that an
item is added to the cart. Grove Collective attempts to avoid this dark
pattern by clearly communicating through the interface and email reminders.
Yet, no matter how well the divergence from the norm is communicated, users
will still feel that their expectations are violated.

In contrast, StitchFix avoids breaking users’ expectations through the way it
communicates its concepts. When a user opens the site, there is no option to
add items to a cart. Instead, they are directed to the style quiz, which has
an interface clearly communicating that stylists will be selecting pieces for
the user. As a result, users do not form an observed concept that includes
Shopping Cart, and instead correctly observe that a different concept (which
we might call PersonalShopper to connote how it acts on behalf of the user) is
in use. By immediately establishing that they are using a different concept
from other shopping sites, StitchFix can implement similar functionality to
Grove Collective, but without forming expectations they aren’t able to
fulfill.

We can compare Grove Collective and StitchFix both procedurally and
substantively to understand how users perceive them differently. Procedurally,
Grove Collective causes most users to expect a Shopping Cart, which they
implement in a non-standard way. Users who find Grove Collective helpful were
likely able to discern that the underlying concept was indeed not a Shopping
Cart, and form their expectations accordingly. StitchFix communicates the
PersonalShopper concept, and helps users understand what they should expect it
to do. Users who still interpret the concept as a Shopping Cart may also
experience a divergence in observed concept and expected concept. But from a
substantive perspective, this does not necessarily mean that they will
perceive this as a dark pattern. For users who highly value the convenience
that the automation brings them, they will feel the divergence is to their
benefit—in other words, it is in line with their values. However, StitchFix
users whose observed concept is Shopping Cart, and who do not feel the
automation is helpful to them, will perceive StitchFix negatively.

By identifying the parts of an application that violate a standard / expected
concept and conflict with users’ values and priorities, we can generate
solutions. In the case of Grove Collaborative, we identify that the deviation
from the standard Shopping Cart comes from the shop having access to add(),
and that some users find that this benefits the company at their expense. This
can be straightforwardly addressed by only allowing the user to initiate the
add() action. However, the shop having access to add() is a fundamental part
of Grove’s value proposition—Grove seeks to automate and remove the burden of
shopping for commonly needed household items. This mismatch between a design
goal and the standard concept’s functionality is indicative that the concept
is not a good fit for the application’s needs. To address this, Grove’s
functionality should be introduced as a distinct concept from Shopping Cart,
as StitchFix has done. By establishing a separate concept like
PersonalShopper, rather than using standard Shopping Cart UI elements, Grove
would no longer be fighting against the expectations users bring from their
past Shopping Cart experiences.

##  7\. Discussion: Implications for Stakeholders

###  7.1. Designers

In addition to checking for surface-level UI misrepresentations, designers
should also analyze their designs in terms of user expectations and consider
the normative implications when designs diverge from expectations. Concept
theory helps designers perform procedural and substantive analysis in a
systematic way, grounded in concrete units of functionality. Designers can
apply procedural theory to consider what concepts their design choices
communicate, and how those concepts align with users’ expectations. Designers
can apply substantive theory to consider how their design choices uphold or
neglect different values, and whether those choices reproduce structural
inequality. Designers can advocate for their organizations to create concept
catalogs, establishing a shared basis for the design standards and shared
language for evaluating potential designs.

###  7.2. Policymakers

Policymakers should continue to advocate for ethical design, but must
acknowledge that it is impossible to regulate away all dark patterns. This is
because dark patterns have procedural and substantive aspects, and regulation
only addresses procedural considerations. Policymakers can focus initially on
preventing more straightforward cases of deceptive interface design that
misrepresents the underlying functionality of an application. They can build
on this work by identifying and defining standard design expectations in
sensitive domains like privacy, then enforcing those expectations. However,
while some standard concepts should be defined in regulation for the public
interest, some can be left to industry norms and standards. Rather than
creating extensive documents that attempt to outline all the dark patterns to
avoid, policymakers can encourage companies to establish and share positive
guidelines for standard concepts.

###  7.3. Researchers

Our work suggests that the current taxonomy-based approach to dark patterns is
limited, and that a shift in approach is called for. Rather than building new
taxonomies, we encourage researchers to focus instead on the recurrent
functional patterns (that is, the concepts) that characterize apps, the
expectations associated with them, and the substantive ethical questions that
can determine when deviations from expectation are acceptable.

##  Appendix A appendix

Below we define the guidelines for mapping the shopping cart concept to the
user interface, to be used in combination with the mapping principles from
Section 5.3. Interface mapping examples from existing applications can be seen
in Figure 5.3.

items, price, quantity

  * •

Each item in items should be displayed in the cart UI along with its price and
quantity

  * •

The price of each item in items should be the same as its price in the
corresponding catalog

  * •

The quantity of each item in items should be less than or equal to the stock
available for that item in the corresponding catalog

subtotal

  * •

The subtotal should be displayed in the cart UI.

  * •

The conventional meaning of the word “Subtotal” is the total cost of the
merchandise before shipping, taxes, and fees. Thus, no other costs in the UI
should be labeled “Subtotal” – if additional costs are added, the new totals
should be labeled “Total”, “Estimated Total”, “PreTax Order Total”, or a
reasonable variation that accurately conveys the fees included in the
estimate.

add()

  * •

There should be an add() button for each item in the corresponding catalog,
and the representation should be consistent.

  * •

In addition to managing inventory, the sync between add() and
removeFromStock() in catalog prevents users from adding items that are out of
stock. An error message should be displayed if a user tries to add() an item
that is out of stock, so that the sync is clear to the user. Alternatively,
the add() button can be unclickable if the item is out of stock.

changeQuantity()

  * •

There should be a changeQuantity() button for each item in items, and the
representation should be consistent

  * •

Common instantiations of the UI element representing changeQuantity() are a
drop down, a text box with -/+ steppers, or a text box with up/down arrow
steppers. The UI should prevent users from choosing a quantity greater than
the actual stock available for that item in the catalog. For example, the
dropdown could only contain quantities up to the stock available, or the
steppers could stop working when the quantity reached is 0 or stock.

  * •

Changing the quantity of the item to 0, if allowed, should be synced with the
action to remove() the item from the cart, and the UI should reflect this
change

remove()

  * •

There should be a remove() button for each item in items, and the
representation should be consistent.

checkout()

  * •

There should be a checkout() button in the cart UI.

  * •

The checkout experience in a typical e-commerce website is implemented by
synchronizing the fulfill() action in order with checkout(). In the cart
definition, checkout() is functionally equivalent to emptying the cart, but is
named checkout() to emphasize that its primary use is in the checkout
experience. However, it is acceptable to include an additional UI element to
allow users to empty their cart, which also uses checkout() under the hood to
clear items.

(A)

(B)

(C)

Figure 7. Examples of shopping cart user interfaces. (A) items, quantity,
price, remove() on amazon.com (B) subtotal, checkout() on amazon.com (C) add()
on amazon.com

Screenshots of the Amazon user interface. (A) A sample shopping cart with two
items added. (B) Underneath the subtotal is a button to proceed to checkout.
(C) Underneath the price of the item are add to cart and buy now buttons.

Synchronizations

Syncs with Catalog

  * •

The p parameter to add() in cart should be price of the item in the catalog

  * •

When an item is added to or removed from the cart, the corresponding catalog
stock should be updated.

    * –

add() → removeFromStock()

    * –

changeQuantity() → addToStock(), removeFromStock()

    * –

remove() → addToStock()

Syncs with Order

  * •

When a cart is checked out, an order should be created

    * –

create() ← checkout()

Syncs with Coupon

  * •

When a coupon is applied to an item in the cart, the price of that item should
be updated

    * –

changePrice() ← apply()

## References

  * (1)
  * noa ([n. d.]) [n. d.].  GoodUX.  https://goodux.appcues.com
  * noa (2021) 2021\.  Similarweb Data Methodology.  https://support.similarweb.com/hc/en-us/articles/360001631538-Similarweb-Data-Methodology
  * noa (2022a) 2022a.  Bringing Dark Patterns to Light.  https://www.ftc.gov/reports/bringing-dark-patterns-light
  * noa (2022b) 2022b.  _Evidence review of Online Choice Architecture and consumer and competition harm_.  Technical Report. Competition & Markets Authority, UK.  https://www.gov.uk/government/publications/online-choice-architecture-how-digital-design-can-harm-competition-and-consumers/evidence-review-of-online-choice-architecture-and-consumer-and-competition-harm
  * noa (2023a) 2023a.  Fashion Styling: How Does Stitch Fix Work? | Stitch Fix.  https://www.stitchfix.com/how-it-works
  * noa (2023b) 2023b.  Grove Reviews.  https://www.sitejabber.com/reviews/grove.co
  * noa (2023c) 2023c.  Human Interface Guidelines.  https://developer.apple.com/design/human-interface-guidelines/
  * noa (2023d) 2023d.  Material Design.  https://m3.material.io/
  * noa (2023e) 2023e.  Stitch Fix Reviews.  https://au.sitejabber.com/reviews/stitchfix.com
  * noa (2023f) 2023f.  Top Websites Ranking.  https://www.similarweb.com/top-websites/e-commerce-and-shopping/
  * noa (2023g) 2023g.  UI-Patterns.com.  https://ui-patterns.com/
  * noa (2023h) 2023h.  UXArchive - Made by Waldo.  http://uxarchive.com/
  * noa (2023i) 2023i.  What is Grove Collaborative?  https://community.grove.co/hc/en-us/articles/204842349-What-is-Grove-Collaborative-
  * Beaudouin-Lafon (2004) Michel Beaudouin-Lafon. 2004\.  Designing interaction, not interfaces. In _Proceedings of the working conference on Advanced visual interfaces - AVI ’04_. ACM Press, Gallipoli, Italy, 15.  https://doi.org/10.1145/989863.989865
  * Brignull (2010) Harry Brignull. 2010\.  Dark patterns.  https://old.deceptive.design/main_page/index.html
  * Brignull (2013) Harry Brignull. 2013\.  Dark Patterns and Sports Direct on BBC Watchdog.  https://90percentofeverything.com/2013/10/21/dark-patterns-and-sports-direct-on-bbc-watchdog/
  * Brignull et al. (2023) H Brignull, M Leiser, C Santos, and K Doshi. 2023\.  Deceptive patterns – user interfaces designed to trick you.  https://www.deceptive.design/
  * Bösch et al. (2016) Christoph Bösch, Benjamin Erb, Frank Kargl, Henning Kopp, and Stefan Pfattheicher. 2016.  Tales from the Dark Side: Privacy Dark Strategies and Privacy Dark Patterns.  _Proceedings on Privacy Enhancing Technologies_ 2016, 4 (Oct. 2016), 237–254.  https://doi.org/10.1515/popets-2016-0038
  * cohost.org/hthrflwrs [@HTHRFLWRS] (2023) cohost.org/hthrflwrs [@HTHRFLWRS]. 2023\.  duolingo uses a ton of predatory/addictive dark ux to keep people engaged with their app, but the thing they’re engaging with is learning a new language, so it’s actually fine.  https://twitter.com/HTHRFLWRS/status/1631303949864034304
  * Commission et al. (2022) European Commission, Directorate-General for Justice, Consumers, F Lupiáñez-Villanueva, A Boluda, F Bogliacino, G Liva, L Lechardoy, and T Rodríguez de las Heras Ballell. 2022\.  _Behavioural study on unfair commercial practices in the digital environment : dark patterns and manipulative personalisation : final report_.  Publications Office of the European Union.  https://doi.org/doi/10.2838/859030
  * Costanza-Chock (2020) Sasha Costanza-Chock. 2020\.  _Design Justice: Towards an Intersectional Feminist Framework for Design Theory and Practice_.  MIT Press, Cambridge, MA.  https://papers.ssrn.com/abstract=3189696
  * Council (2018) Norweigan Consumer Council. 2018\.  Deceived by design, how tech companies use dark patterns to discourage us from exercising our rights to privacy.  https://fil.forbrukerradet.no/wp-content/uploads/2018/06/2018-06-27-deceived-by-design-final.pdf
  * Di Geronimo et al. (2020) Linda Di Geronimo, Larissa Braz, Enrico Fregnan, Fabio Palomba, and Alberto Bacchelli. 2020.  UI Dark Patterns and Where to Find Them: A Study on Mobile Applications and User Perception. In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_. ACM, Honolulu HI USA, 1–14.  https://doi.org/10.1145/3313831.3376600
  * Friedman et al. (2007) Batya Friedman, Peter H. Kahn, Jr, and and Alan Borning. 2007\.  Value Sensitive Design and Information Systems.  In _Human-computer Interaction and Management Information Systems: Foundations_. Routledge.  Num Pages: 25.
  * Gamma et al. (1994) Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. 1994.  _Design Patterns: Elements of Reusable Object-Oriented Software_.
  * Gaudel (1994) M.-C. Gaudel. 1994\.  Formal specification techniques. In _Proceedings of 16th International Conference on Software Engineering_. 223–227.  https://doi.org/10.1109/ICSE.1994.296781 ISSN: 0270-5257.
  * Gray (2022) Colin M. Gray. 2022\.  UXP2 Dark Patterns.  https://darkpatterns.uxp2.com/
  * Gray et al. (2018) Colin M. Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L. Toombs. 2018.  The Dark (Patterns) Side of UX Design. In _Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems_. ACM, Montreal QC Canada, 1–14.  https://doi.org/10.1145/3173574.3174108
  * Gray et al. (2023) Colin M. Gray, Cristiana Santos, and Nataliia Bielova. 2023\.  Towards a Preliminary Ontology of Dark Patterns Knowledge. In _Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems_ _(CHI EA ’23)_. Association for Computing Machinery, New York, NY, USA, 1–9.  https://doi.org/10.1145/3544549.3585676
  * Gunawan et al. (2021) Johanna Gunawan, Amogh Pradeep, David Choffnes, Woodrow Hartzog, and Christo Wilson. 2021\.  A Comparative Study of Dark Patterns Across Web and Mobile Modalities.  _Proceedings of the ACM on Human-Computer Interaction_ 5, CSCW2 (Oct. 2021), 1–29.  https://doi.org/10.1145/3479521
  * Hamraie (2013) Aimi Hamraie. 2013\.  Designing Collective Access: A Feminist Disability Theory of Universal Design.  _Disability Studies Quarterly_ 33, 4 (Sept. 2013).  https://doi.org/10.18061/dsq.v33i4.3871 Number: 4.
  * Hidaka et al. (2023) Shun Hidaka, Sota Kobuki, Mizuki Watanabe, and Katie Seaborn. 2023\.  Linguistic Dead-Ends and Alphabet Soup: Finding Dark Patterns in Japanese Apps. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_. ACM, Hamburg Germany, 1–13.  https://doi.org/10.1145/3544548.3580942
  * Hoffmann (2019) Anna Lauren Hoffmann. 2019\.  Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse.  _Information, Communication & Society_ 22, 7 (June 2019), 900–915.  https://doi.org/10.1080/1369118X.2019.1573912
  * Jackson (2021) Daniel Jackson. 2021\.  _The Essence of Software: Why Concepts Matter for Great Design_.  Princeton University Press.
  * Jansen et al. (2022) Bernard J. Jansen, Soon-gyo Jung, and Joni Salminen. 2022\.  Measuring user interactions with websites: A comparison of two industry standard analytics approaches using data of 86 websites.  _PLOS ONE_ 17, 5 (May 2022), e0268212.  https://doi.org/10.1371/journal.pone.0268212 Publisher: Public Library of Science.
  * Kowalczyk et al. (2023) Monica Kowalczyk, Johanna T. Gunawan, David Choffnes, Daniel J Dubois, Woodrow Hartzog, and Christo Wilson. 2023.  Understanding Dark Patterns in Home IoT Devices. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_. ACM, Hamburg Germany, 1–27.  https://doi.org/10.1145/3544548.3581432
  * Liskov and Zilles (1974) Barbara Liskov and Stephen Zilles. 1974.  Programming with abstract data types. In _Proceedings of the ACM SIGPLAN symposium on Very high level languages_. Association for Computing Machinery, New York, NY, USA, 50–59.  https://doi.org/10.1145/800233.807045
  * Maier and Harr (2020) Maximilian Maier and Rikard Harr. 2020.  Dark Design Patterns: An End-User Perspective.  _Human Technology_ 16, 2 (Aug. 2020), 170–199.  https://doi.org/10.17011/ht/urn.202008245641
  * Mathur et al. (2019) Arunesh Mathur, Gunes Acar, Michael J. Friedman, Elena Lucherini, Jonathan Mayer, Marshini Chetty, and Arvind Narayanan. 2019\.  Dark Patterns at Scale: Findings from a Crawl of 11K Shopping Websites.  _Proceedings of the ACM on Human-Computer Interaction_ 3, CSCW (Nov. 2019), 1–32.  https://doi.org/10.1145/3359183 arXiv:1907.07032 [cs].
  * Mathur et al. (2021) Arunesh Mathur, Jonathan Mayer, and Mihir Kshirsagar. 2021\.  What Makes a Dark Pattern… Dark? Design Attributes, Normative Considerations, and Measurement Methods. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_. 1–18.  https://doi.org/10.1145/3411764.3445610 arXiv:2101.04843 [cs].
  * Mildner et al. (2023) Thomas Mildner, Gian-Luca Savino, Philip R. Doyle, Benjamin R. Cowan, and Rainer Malaka. 2023\.  About Engaging and Governing Strategies: A Thematic Analysis of Dark Patterns in Social Networking Services. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_. ACM, Hamburg Germany, 1–15.  https://doi.org/10.1145/3544548.3580695
  * Nielson (2020) Jakob Nielson. 2020\.  10 Usability Heuristics for User Interface Design.  https://www.nngroup.com/articles/ten-usability-heuristics/
  * Norman (1998) Don Norman. 1998\.  _The Design of Everyday Things_.  The MIT Press.  https://mitpress.mit.edu/9780262640374/the-design-of-everyday-things/
  * Renaud and Zimmermann (2018) Karen Renaud and Verena Zimmermann. 2018.  Ethical guidelines for nudging in information security & privacy.  _International Journal of Human-Computer Studies_ 120 (Dec. 2018), 22–35.  https://doi.org/10.1016/j.ijhcs.2018.05.011
  * Rossi and Bongard-Blanchy (2021) Arianna Rossi and Kerstin Bongard-Blanchy. 2021.  All in one stroke? Intervention Spaces for Dark Patterns.  http://arxiv.org/abs/2103.08483 arXiv:2103.08483 [cs].
  * Soe et al. (2020) Than Htut Soe, Oda Elise Nordberg, Frode Guribye, and Marija Slavkovik. 2020.  Circumvention by design - dark patterns in cookie consent for online news outlets. In _Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society_ _(NordiCHI ’20)_. Association for Computing Machinery, New York, NY, USA, 1–12.  https://doi.org/10.1145/3419249.3420132
  * Tognazzini (2014) Bruce Tognazzini. 2014\.  First Principles of Interaction Design (Revised & Expanded).  https://asktog.com/atc/principles-of-interaction-design/
  * u/frankie_co_su (2020) u/frankie_co_su. 2020\.  There’s just too much wrong with Duolingo - bidding farewell : duolingo.  https://www.reddit.com/r/duolingo/comments/isfbex/theres_just_too_much_wrong_with_duolingo_bidding/
  * Utz et al. (2019) Christine Utz, Martin Degeling, Sascha Fahl, Florian Schaub, and Thorsten Holz. 2019.  (Un)informed Consent: Studying GDPR Consent Notices in the Field. In _Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security_. 973–990.  https://doi.org/10.1145/3319535.3354212 arXiv:1909.02638 [cs].
  * Winner (1980) Langdon Winner. 1980\.  Do artifacts have politics?  _Daedalus_ 109, 1 (1980), 121–136.
  * Zagal et al. (2013) José P Zagal, Staffan Björk, and Chris Lewis. 2013\.  Dark Patterns in the Design of Games.  (2013).
  * Zong and Matias (2022) Jonathan Zong and J. Nathan Matias. 2022.  Bartleby: Procedural and Substantive Ethics in the Design of Research Ethics Systems.  https://doi.org/10.1177/20563051221077021