# Making Markets for Information Security:
The Role of Online Platforms in Bug Bounty Programs

Johannes Wachs
Vienna University of Economics and Business
Complexity Science Hub Vienna
<EMAIL_ADDRESS>
###### Abstract

Security is an essential cornerstone of functioning digital marketplaces and
communities. If users doubt that data shared online will remain secure, they
will withdraw from platforms. Even when firms take these risks seriously,
security expertise is expensive and vulnerabilities are diverse in nature.
Increasingly, firms and governments are turning to bug bounty programs (BBPs)
to crowdsource their cybersecurity, in which they pay individuals for
reporting vulnerabilities in their systems. And while the use of BBPs has
grown significantly in recent years, research on the actors in this market and
their incentives remains limited. Using the lens of transaction cost
economics, this paper examines the incentives of firms and researchers
(sometimes called hackers) participating in BBPs. We study the crucial role
that centralized platforms that organize BBPs play in this emerging market. We
carry out an analysis of the HackerOne BBP platform, using a novel dataset on
over 14,000 researchers reporting over 125,000 public vulnerabilities to over
500 firms from 2014 to the end of 2021. We outline how platforms like
HackerOne make a market for information security vulnerabilities by reducing
information asymmetries and their associated transaction costs.

Keywords: Online Markets, Information Security, Crowdsourcing, Trust, Hacking

## Introduction

Information security plays a key role in the digital economy, especially on
sites and platforms where users provide private information or exchange money
for goods and services. These online platforms seem to work best when they
establish trust with their users, meaning that users feel secure because of
features of the system the platform operates, rather than the behavior of
other users (Pavlou and Gefen, 2004). As more users participate in these
online platforms and marketplaces, the security stakes rise and systems are
coming under increasing pressure from malicious actors that seek to steal
information or defraud users, leading to a variety of harmful outcomes
(Agrafiotis et al., 2018). And online platforms are not the only domain in
which information security has become a major issue. Indeed, as digitization
extends to a wider variety of consumer goods and services from automobiles
(Svangren et al., 2017) to e-government (Zhao and Zhao, 2010), the scope of
potential vulnerabilities has only increased. Vulnerabilities often have
externalities, affecting other firms and individuals beyond the confines of a
single firm (Anderson, 2001; Gordon et al., 2014; Ahmed, 2021), suggesting
that investment in information security will be suboptimal from a social
welfare perspective (Gordon et al., 2015). Digital insecurity is known to have
disparate impact across socio-economic groups, compounding disadvantages for
certain groups (Hargittai, 2021). Hacking has even become a geopolitical
issue, as state actors are both victims and perpetrators of attacks (Buchanan,
2020).

How do firms and institutions organize their defense? Traditional in-house
information security functions are often supplemented by consultants who carry
out security audits using techniques such as penetration testing. However, the
growing complexity of software systems makes it difficult for firms to scope
their security needs and for experts to anticipate the many avenues of
potential attacks (Biro et al., 2017; Malladi and Subramanian, 2019). Given
large and growing demand for this kind of knowledge, it has become
increasingly expensive to hire security experts (Libicki et al., 2014).

One alternative attracting attention in recent years is the concept of a bug
bounty program (BBP). Bug bounties are a way of crowdsourcing security in
which a firm pledges to reward outside researchers (colloquially referred to
as hackers) with financial bounties for reporting vulnerabilities they observe
in the firm’s systems. By stating what kind of vulnerability reports they are
interested in and how they handle and potentially reward such reports, firms
can signal to the community of security researchers that they are open to such
exchanges. The core idea is that BBPs, when well implemented, offer a kind of
security audit that harnesses the knowledge and know-how of a much larger and
more diverse group of individuals than any in-house security team or
consulting firm can offer (Maillart et al., 2017).

Historically, BBPs emerged in a decentralized manner, with firms and
institutions offering bounties and setting up rules of engagement on their own
(Goerzen and Coleman, 2022). And while the decentralized market for
information security vulnerabilities seemed to grow significantly in the late
90’s into the 00’s, several issues limited such exchanges between firms and
researchers. For example, a firm may already be aware of a specific
vulnerability when a researcher reports it, and then be unwilling to reward
that researcher. A firm could also underpay a researcher relative to the value
of the exposed vulnerability, especially if they think that this is a one-off
scenario. Even if both parties have good intentions, the inherent complexity
of vulnerability reports makes it difficult to coordinate their resolution and
valuation.

Researchers familiar with Transaction Cost Economics (TCE) (Williamson, 1979)
will recognize these as examples of situations of asymmetric information and
opportunistic behavior in markets. Moreover, as vulnerabilities are highly
specific, complex, and of uncertain frequency, the theory of TCE suggests that
information security will likely be organized within firms, as part of a
hierarchy rather than in a market. Add to these observations the fact that
vulnerabilities, if publicized, can come with significant regulatory and
reputational costs (Goel and Shawky, 2009), it is no wonder that firms often
keep security functions in-house or simply try to ignore the problem.
Similarly, researchers also face several obstacles when participating in
bounty programs. For example, researchers value the publication of their
report after the vulnerability has been resolved as a token of prestige and a
credible demonstration of their skills, which carries significant value on the
labor market (Goerzen and Coleman, 2022; Ellis and Stevens, 2022). A priori it
is unclear which firms will agree to this kind of transparency and under what
conditions, and even if a firm will pay a reasonable reward for such a report.

Overall it seems that information asymmetries are major obstacles to a well-
functioning and large scale market for vulnerabilities. However, the same
forces of digitalization that have increased the importance of information
security in general have also introduced new technologies and organizational
forms for coordinating economic activity. The Electronic Markets Hypothesis
(EMH) of Malone, Yates and Benjamin (1987) predicted that digitalization would
decrease transaction costs in general and expand the scope of activities that
take place in markets, often coordinated by “market makers”. We argue that
this move to the market in information security has taken place in the last
decade, lead by online platforms (Alt and Zimmermann, 2019) (the market
makers) such as HackerOne and BugCrowd. As intermediaries, platforms decrease
transaction costs that would otherwise prevent markets from being viable (Alt,
2017; Nagle et al., 2020). Platforms managing BBPs facilitate this market by
reducing information asymmetries using standardization and reputation systems.

In this article we analyze the market for bug bounties through the perspective
of Transaction Cost Economics, arguing that, in line with the Electronic
Markets Hypothesis, centralized online platforms play a key role in making
such markets possible by reducing information asymmetries. We give an overview
of bug bounty programs and the incentives in this market from both the firm
and researcher perspectives. We then suggest how a centralized platform can
reduce these costs to the extent needed to make a viable market. Using data
from HackerOne, a market leading BBP platform, we present empirical evidence
connecting TCE theory with firm and researcher behavior. We find that
researchers and firms use platform functionalities to solve information
problems (i.e. to signal) and to better capture value of their activity. We
also observe rising specialization of researchers in specific firms over time,
suggesting the emergence of embedded structures in the market. We conclude
with a summary of our contribution, and suggest some open questions in this
area.

## Background and Theoretical Framing

In this section we provide an overview of BBPs, including a brief history. We
highlight information problems that firms and researchers face when
participating in bug bounties. We then theorize how centralized platforms for
managing BBPs help resolve these problems. We then develop a set of hypotheses
about these theories, which we can be tested using empirical data we have
collected from the HackerOne platform.

### Background on Bug Bounty Programs

The first BBPs were established, perhaps unsurprisingly, by firms and
institutions working in the software industry (Goerzen and Coleman, 2022).
Besides the fact that software is the natural arena of digital security, the
industry has a long tradition of decentralized, crowd-based collaboration
(LaToza and Van Der Hoek, 2015), for instance in open source software (OSS).
Indeed, recent work has adapted Eric Raymond’s maxim on the effectiveness of
OSS development, that “with enough eyeballs, all bugs are shallow” (Raymond,
1999) to the context of information security vulnerabilities via BBPs
(Maillart et al., 2017). In this framing, BBPs are an organizational framework
to crowdsource an important problem, and crowdsourcing itself is an effective
way to harness diverse human resources and collective intelligence (Agrawal et
al., 2014; Riedl et al., 2021; Wachs and Vedres, 2021).

The first such programs emerged out of a volatile period in the information
security community in the 1990s (Goerzen and Coleman, 2022). Researchers and
hackers investigating widely used software for vulnerabilities became
increasingly frustrated with how firms responded to their findings either by
ignoring them or reporting them to the authorities. The Bugtraq mailing list,
started in 1993, became a place to fully disclose and document such
vulnerabilities. Goerzen and Coleman (2022) argue that researchers and hackers
operating outside institutional environments began participating in this
mailing list to establish legitimacy for themselves and their community, and
to exchange ideas with insiders. Though highly controversial at first,
collective efforts like Bugtraq demonstrated both the effectiveness of
crowdsourced insights into security and that many “hackers” could be
trustworthy contributors.

In parallel, leading tech firms established the first BBPs (Ellis and Stevens,
2022). The term bug bounty program was coined at Netscape in 1995, when the
company launched such a program with an initial budget of $50,000. However,
one condition of Netscape’s BBP was that rewarded vulnerabilities would not be
publicly disclosed. Indeed, as Ellis and Stevens (2022) document, Netscape had
experienced bad publicity when several vulnerabilities in its browser were
published to Bugtraq. In this way, Netscape was attempting to limit the
uncertainty and potential costs of vulnerabilities: a bounty paid to a
researcher was likely cheaper in the long-term than a disclosed major
vulnerability on Bugtraq. This highlights a major tension in BBPs: researchers
often want a report to be disclosed for a variety of reasons, while firms tend
to want to keep them secret.

It took some time for BBPs to become more widely adopted. Google and Facebook
launched their own programs only in 2010 and 2011, respectively (Ellis and
Stevens, 2022). And while by the 2020s some companies host their own BBPs, the
idea became really widespread, and used by smaller firms, only after the
establishment of centralized platforms managing BBPs like HackerOne and
Bugcrowd. Among private or invitation-only BBPs, NDA agreements are common and
disclosure is rare. Activity on these centralized platforms is signficant:
HackerOne reported that firms had paid out over $36 million in bounties in
2021 (HackerOne, 2021), with the median “critical” bug netting a researcher
around $3,000.

#### Researcher Participation in BBPs

Though a full investigation of why individuals participate in BBPs as
researchers is beyond the scope of this article, we briefly consider some
motivations for participation here. Our emphasis is on how motivations come
into conflict with the those of firms on the other side of a potential
transaction or how lack of information may be a barrier to continued
participation.

Around 40% of researchers participating in a 2019 HackerOne survey claim they
participate in BBPs in order to learn and build a career in information
security (HackerOne, 2019). Roughly the same share of respondents say they
hack for fun (13%) as for the money (14%) - though this survey is not based on
a representative sample. HackerOne materials emphasize that BBPs offer
flexible work opportunities, where individuals can decide how much they work
and with which companies they work. This and other aspects of hacking closely
resemble the characterization of “gig work” or the “gig economy” (Graham et
al., 2017; Ellis and Stevens, 2022). Many researchers do work only part time
on BBPs (Hata et al., 2017), either supplementing their main income or as part
of a transitory stage in their careers.

In practice researchers face several issues resulting from a lack of
information. For example, a vulnerability that has already been reported but
not yet fixed is rarely rewarded (Zhao et al., 2017), creating a significant
amount of wasted effort. While firms attempt to pre-commit to certain kinds of
payments depending on the kind and severity of the vulnerability, many
vulnerabilities are unique in some way and often quite complex. This makes it
unclear how a vulnerability will be mapped to the menu of rewards, and can
lead to a researcher being disappointed in their reward. Researchers can
exchange information about how well firms treat them on message boards and
community forums, but this introduces significant searching costs.

Finally, as discussed above, researchers are often interested in getting their
vulnerability reports disclosed so that they can demonstrate their
accomplishments. When building a career and resume in the information security
sector, a credible record of having hacked well-known tech companies can go a
long way (Ellis and Stevens, 2022). These kinds of credentials can substitute
for formal degrees, and may be even more valuable for individuals from
developing countries. Firms can often simply refuse to disclose a report, and
may be more likely to do so if the vulnerability is embarrassing or hints at
other potential vulnerabilities. Disclosure is important from a more general
perspective as well. Goerzen and Coleman highlight several reasons (2022):

  * •

Disclosure opens knowledge about a vulnerability to a wide audience.

  * •

Repeated disclosure creates a space for both sides to share information and
arrive to common standards which can improve communication in subsequent
interactions (Blind, 2016).

  * •

It resolve issues of credit and “ownership” of vulnerabilities.

  * •

Competition among researchers creates a way to define reputation and to build
CVs.

  * •

Good examples demonstrate the social value of “hacking” or information
security research in general.

Little of the value of these points in favor of vulnerability disclosure is
captured directly by the firm that has been hacked. Rather, disclosure creates
many positive externalities, enjoyed by the community as a whole. As such, it
is likely that vulnerabilities are disclosed at a socially sub-optimal rate.

#### Firm Participation in BBPs

When starting a BBP, firms tend to set specific rules of engagement (Laszka et
al., 2018), highlighting which parts of their codebase are in scope, and
committing to actions in response, for instance the level of payment and
response time. When responding to individual vulnerability reports, firms need
to decide whether the vulnerability is relevant and valid, and if so - how to
reward the researcher. The danger in undercutting a researcher is that word
travels and that other researchers may not be interested in investigating
one’s system. Firms have to weigh this risk against the size of their budget
for the BBP.

After a vulnerability is resolved, the firm should decide whether or not to
disclose the vulnerability and the interaction with the researcher. Besides
appeasing the researcher, firms may signal to other researchers that they are
approachable and easy to deal with. On the other hand, firms may not want to
admit having these mistakes in their systems - as it may cause them to suffer
reputational damage and lose esteem in the eyes of users and customers.

#### Online Platforms as Market Makers for BBPs

These tensions on both the firm and information security researcher side
impose significant costs on their transactions, and likely limit the potential
of decentralized BBPs in which each firm organizes its own program. Indeed,
the vulnerabilities as assets are highly specific, relatively infrequent, and
involve significant uncertainties. While vulnerabilities are can be
categorized in general, the particular details of a vulnerability within a
system and its consequences tend to be unique. A researcher may report
multiple vulnerabilities to a firm, but these will likely come at irregular
intervals. And finally, the process of reporting a vulnerability carries
significant uncertainty for both researcher and firm. A firm may reject a
vulnerability as irrelevant or undervalue it relative to the researcher’s
expectations, or may simply ignore the researcher, or in extreme cases
consider the vulnerability report a trespass a report the researcher to the
authorities (Goerzen and Coleman, 2022). Firm-organized bug bounty programs
that solicit reports often try to address this issue by setting rules of
engagement and a menu of rewards. However, because vulnerabilities are so
specific, their valuation always requires some coordination.

Transaction Cost Economics suggests that transactions with these
characteristics are likely to be carried out within firms (Williamson, 1979;
Alt, 2017). This likely explains the relatively slow proliferation of BBPs in
the early 2000s, before the emergence of centralized platforms like HackerOne
and Bugcrowd that manage bounties for many firms. These online platforms
reduce information asymmetries and other frictional costs associated with the
transaction of specific, infrequent, and uncertain assets. They achieve this
by introducing reputation mechanisms and providing standardized procedures.

Nagle, Seamans, and Tadelis (2020) highlight reputation mechanisms as a key
mechanism by which online platforms reduce transaction costs and make markets.
On HackerOne, researchers and firms have profiles that report detailed
information about their activity on the platform. Statistics on a firm’s
responsiveness to reports, and the sizes of their bounties are public
information. Firm profiles also set the scope of their assets they would pay
bounties on, and often state how much they pay for vulnerabilities by
severity. Though in the end the judgement of the severity of a vulnerability
remains subjective, providing examples of vulnerabilities meeting different
grades of severity (i.e. if a user’s private information can be captured using
a vulnerability) decreases the degree of specificity in the market.

Besides listing the number of vulnerabilities they have reported to various
companies, researchers collect reputation points, credits, and badges on their
profile pages. These elements have value as credentials, and allow researchers
to build a track record demonstrating expertise. Researcher profiles can also
link to an individual’s personal website and to other online platforms like
LinkedIn, Twitter, or GitHub. This makes the user profile a hub through which
headhunters and hiring managers can learn more about them. Such functions play
an increasingly important role in digital labor markets (Marlow and Dabbish,
2013; Papoutsoglou et al., 2021), and allow researchers to capture ever more
value from their activity on the site.

HackerOne also facilitates the market for vulnerabilities by standardizing the
process by which vulnerabilities are reported and communication is carried
out. Several firms participating on HackerOne actually hire HackerOne
employees to manage their BBPs. HackerOne also provides general guidelines,
suggests best practices, and moderates disputes. The platform also serves as a
record of activity for both firms and researchers, and gives both kinds of
actors the chance to learn from the activities of others.

#### Hypotheses

Though the case that HackerOne facilitates the market for vulnerabilities by
reducing information asymmetries and allowing researchers to capture greater
value from their activity is in line with the predictions of TCE and the EMH,
we now propose a series of empirical hypotheses about activity on the platform
to verify these connections. Specifically, we ask if users and firms respond
to information incentives, and if behaviors such as specialization and
embeddedness of market participants emerge over time.

First we probe whether researchers actually move to capture labor market value
of their activity on the site by connecting their profiles on HackerOne to
their other avatars on the web.

> H1: More active researchers will link their profiles to their other
> presences on the web more frequently, especially if they have disclosed
> vulnerability reports.

Second, we propose that firms will provide additional signals and information
about their behavior in their first interactions on the platform by disclosing
more reports.

> H2: Firms will be more likely to disclose vulnerabilities early on in their
> history on the platform.

Finally, we suggest that researchers will tend to specialize and work more
frequently with specific companies as time goes on, reflecting improved market
matching owing to the proliferation of information on the platform.

> H3: Researchers will tend to focus their efforts on fewer firms over time.

## Empirical Analysis

In this section we carry out an empirical analysis of data from HackerOne. We
first describe our data and its source. We present descriptive statistics
about researchers, firms, and reported vulnerabilities. We then analyze the
observed behavior of firms and researchers. We first observe a relationship
between researcher activity and the extent to which they link to external
platforms. We then investigate whether firms tend to disclose vulnerabilities
earlier in their participation on the platform using a regression modeling
framework. Finally we study the specialization behavior of researchers,
measuring their tendency to either diversify between companies or focus on
specific companies.

### Data

We collect data from HackerOne, one of the largest online platforms managing
BBPs. Vulnerabilities, referred to as hacks, are the main unit of interaction
on HackerOne. Tracing the stream of publicly listed hacks (around 20% of the
total vulnerabilities managed by HackerOne (HackerOne, 2019)) back in time to
the launch of the site, we collect data on 124,172 vulnerabilities reported by
researchers to firms from January 2014 to November 2021. We record the latest
time of the interaction, the researcher and firm involved, whether the
vulnerability was disclosed, and, if announced, how much money the researcher
received for the report. Only 10,966 ($\approx 9\%$) of vulnerabilities were
disclosed. 24,091 ($\approx 19\%$) of reported vulnerabilities lead to a
visible cash payment, summing up to just over $25 million. These cash payments
were highly heterogeneous, with a mean of 1,047, a standard deviation of
2,600, and a median of 300.

Our dataset includes 14,371 researchers. We also visited each of their profile
pages and recorded information about their registration date on HackerOne and
whether or not they link to other platforms. We also record the free-text
location provided by 3,700 researchers. Geolocating these locations using the
Bing Maps API, we could infer a country-level location for 3,354 of them. The
top six countries were India (704, 21%), US (520, 16%), UK (148, 4%), Russia
(131, 4%), Germany (116, 3%) and Brazil (82, 2%). This ranking is highly
similar to the one reported in the 2019 HackerOne survey (HackerOne, 2019), in
which India, the US and Russia were reported as the top three countries. We
note that the geographic distribution of HackerOne researchers seems to more
closely resemble the geographic distribution of remote/gig workers (Braesemann
et al., 2021) than that of software developers (Wachs et al., 2022).

We found 502 firms in the history of public vulnerabilities. These contain a
wide range of entities, from Fortune 500 mainstays like General Motors and
Starbucks to technology companies like Adobe, Facebook, and Spotify, to
smaller startups. Though we refer to firms, non-business entities are also
present including public sectors actors like the Department of Defense and
non-profit organizations and open source software projects like Ruby.

### Researcher Profiles

It has been suggested that a track record of successful vulnerability reports
has significant value on the information security labor market (Mumtaz et al.,
2020; Ellis and Stevens, 2022). To realize this value, individuals must be
able to credibly link their activity on platforms like HackerOne to their
other identities on the web, for instance on coding platforms like GitHub,
professional platforms like LinkedIn, or social platforms like Twitter. We
collected data from researcher profile pages on HackerOne, recording which
other platforms they link to. In Table 1, we report the share of researchers
linking to an external platform by the number of reports they submitted.
Specifically we look for any links to personal websites, GitHub, Bugcrowd,
Gitlab, Linkedin, or Twitter. We observe that while only one in three
researchers in the whole population of researchers link to an external
platform, nearly two in three that have made at least 10 vulnerability reports
(59%) do so. Researchers with at least 2 disclosed reports share an external
link at the same rate (59%). 80% of researchers with at least 10 disclosed
reports have an external link.

$\geq K$ Vuln. | Share of Researchers Linking to External Platform
---|---
1 | 33%
2 | 42%
5 | 52%
10 | 59%
$\geq K$ Disclosed Vuln. | Share of Researchers Linking to External Platform
1 | 33%
2 | 59%
5 | 71%
10 | 79%
Table 1: The share of researchers with at least $K$ credited vulnerability
reports sharing links to external platforms, repeated for $K$ disclosed (i.e.
publicly visible) reports. More active researchers are more likely to link to
their avatars on other platforms, and do so at a greater rate if they have
more disclosed reports to their name. We count links to personal websites,
GitHub, Bugcrowd, Gitlab, Linkedin, or Twitter.

We observe a relationship that is perhaps not surprising: the more active a
researcher is on HackerOne, the more likely they are to link to other
platforms. That this likelihood sharply increases when a researcher has
disclosed reports suggests that there are information effects at play.
Specifically, disclosed vulnerability reports may have significant value as
credentials, demonstrating in greater detail what a researcher is capable of,
or for bragging rights (Ellis and Stevens, 2022). Researchers seem to move to
capture this extra value by linking to their other online presences. The most
prolific researchers tend to make themselves more visible.

### Disclosure

As discussed above, disclosed vulnerability reports do not only explain the
vulnerability and researcher compensation. They offer a transparent look at
the history of communication between researcher and firm for a given
vulnerability. Disclosed vulnerabilities show how quickly firms respond, how
demanding they are for details, and how they treat the reporting researcher.
Yet disclosure of vulnerabilities may have indirect costs for firms. They may
suffer reputational damage, or inadvertently highlight other vulnerabilities
in their systems.

Researchers may be suspicious of a firm new to BBPs or HackerOne. We
hypothesized that a firm will be more likely to disclose vulnerabilities early
on in their history on HackerOne to signal their quality as partners. We test
this hypothesis using the following model for the binary outcome of
vulnerability disclosure:

$\text{Disclosed}=\beta\text{(Early Firm
Vulnerability)}+\alpha_{f}+\gamma_{y}+\mu_{r}+\epsilon$ (1)

The key variable, Early Firm Vulnerability, is a binary indicator that takes
the value 1 when a vulnerability was reported early in the firm’s history on
HackerOne (quantified as the first $K-th$ percentile of the firm’s
vulnerabilities up to the end of 2021. We apply varying thresholds to define
an early vulnerability for a firm (i.e. if it is among the first 10 or 20
percent of vulnerabilities reported to a firm). A positive and statistically
significant estimate of coefficient $\beta$ would indicate that firms are more
likely to disclose early vulnerabilities.

Given that firms may have heterogeneous incentives to disclose or withhold
disclosure, for instance depending on their industry, we include firm fixed-
effects $\alpha_{f}$ to control for time-invariant aspects of firms. We also
control for year of the vulnerability $\gamma_{y}$ to control for changes in
disclosure trends in general. We also include fixed-effects for the reporting
researcher $\mu_{r}$. We cluster standard errors at the researcher-firm level.
This specification controls for unchanging attributes of researchers and
firms, as well as global trends - making it quite conservative.

In our primary specification we fit linear probability model using OLS rather
than a logistic regression because a fair share of the firms and researchers
have either 100% or 0% disclosure rates. A logit model discards such
observations (Caudill et al., 1988). Our findings replicate in a logit model,
which however can only be fit with firm and year fixed-effects. In our primary
specifications, we also exclude data from 2021 as there is often a lag between
a vulnerability’s first announcement in the public HackerOne feed and its
disclosure. Our results are substantively unchanged if 2021 is included.

Dependent Variable: | Vulnerability Disclosed
---|---
| (1) | (2) | (3) | (4)
| LPM | LPM | Logit | Logit
_Vuln. among Firm’s_ |  |  |  |
First 10th Pctile. | 0.022∗∗ |  | 0.458∗∗ |
| (0.008) |  | (0.168) |
First 20th Pctile. |  | 0.021∗∗ |  | 0.421∗∗∗
|  | (0.007) |  | (0.134)
_Fixed-effects_ |  |  |  |
Firm | Yes | Yes | Yes | Yes
Year | Yes | Yes | Yes | Yes
Researcher | Yes | Yes | - | -
_Fit statistics_ |  |  |  |
R2 | 0.52 | 0.52 | - | -
Pseudo R2 | - | - | 0.31 | 0.31
Observations | 96,048 | 96,048 | 79,285 | 79,285
_Researcher-Firm-level clustered standard-errors in parentheses_
_Signif. Codes: ***: 0.01, **: 0.05, *: 0.1_
Table 2: Models predicting whether a vulnerability reported by a researcher to
a firm was publicly disclosed. In both the linear probability model (OLS) and
logistic regression specifications, we observed that vulnerabilities reported
early in a firm’s time on the platform are more likely to be disclosed.

In all four variations of our model we observe a strong effect: firms are
significantly more likely to disclose vulnerability reports early in their
tenure on HackerOne than later. In our preferred specification, the linear
probability model with firm-year-researcher fixed effects, we estimate that
early vulnerabilities are 2.2% more likely to be disclosed. Given that only 9%
of vulnerabilities are disclosed overall, this is a large deviation. We
interpret this result as evidence that firms are eager to signal how it is
like to work with them earlier in their presence on the platform, when the
value disseminating such information is highest. As a firm builds a long track
record, they can be more selective in what vulnerabilities they choose to
disclose.

### Researcher Specialization

Individuals and firms operating in large markets often specialize in certain
sub-markets (Capon et al., 1988). Besides the unilateral returns researchers
may gain from specialization (Nedelkoska et al., 2019; Neffke, 2019), for
instance in a certain class of vulnerabilities or technological stacks used by
multiple firms, they likely gain from bilateral specialization (Di Maggio et
al., 2017). That is to say by repeatedly interacting with the same firms. Such
hybrid (Williamson, 1991) or networked (Powell, 1990; Alt and Klein, 2011)
structures often emerge in markets in which trust is important and assets
remain differentiated. As vulnerability reports are highly specific and the
reactions of firms difficult to predict, we expect that this is the case on
HackerOne.

Such markets are characterized by social connections forged by repeated
interactions (Uzzi, 1996). Indeed, this notion of embeddedness of ties is
known to play a crucial role in online markets with information asymmetries.
In the most sensitive markets, such as those for illegal drugs or other
illicit goods, embeddedness builds a sort of trust without which such markets
could not function (Paquet-Clouston et al., 2018; Duxbury and Haynie, 2021;
Dupont and Lusthaus, 2021). Embeddedness has been observed also in very
traditional offline markets, such as fish markets (Hernandez et al., 2018), in
which goods are specific or have idiosyncratic quality.

We have hypothesized that such embeddedness, manifesting as repeated
interactions between researchers and firms, will increase over the history of
HackerOne. We suggest there are two reasons for this. The first is that highly
active researchers will explore more at the beginning of their activity,
focusing on the most profitable and enjoyable relationships once they have
tested a variety of firms. The second is that information effects will improve
initial market matching over time. Specifically, new researchers observe
previous activity and disclosed reports, and so can more quickly find the
right firms to work with given their skillsets and interests.

We test this idea by calculating the average ratio of researcher strength to
degree per year from 2015 to 2021. Borrowing terms from the networks
literature, researcher strength is defined as the number of vulnerabilities
they are involved in. Researcher degree is defined as the count of the unique
firms they report vulnerabilities to. The ratio of strength to degree then
measures the relative concentration of a researcher on a specific firm. We
report the average ratio across all researchers with at least two
vulnerability reports in each year, along with bootstrapped confidence
intervals in Figure 1.

Figure 1: The average ratio of researcher strength to degree, by year.
Strength is the number of vulnerabilities reported by a researcher, while
degree is the unique number of firms they report vulnerabilities to. A higher
strength to degree ratio indicates researchers are reporting more
vulnerabilities to the same firms. We observe an increase in the relative
frequency of repeated interactions over time. Error bars indicate 95%
bootstrapped confidence intervals.

We observe that the average strength to degree ratio hovers between 2 and 3
early on in the history of the site, then increases to nearly 4 by 2020/2021.
Conservatively, this suggests a roughly 50% increase in the rate at which
researchers specialize on specific firms, over a relatively short time period.

To conclude our analysis we present a practical application of our observation
that researchers often specialize their activity on firms. Specifically, we
visualize the landscape of firms as a network in which nodes are firms and
edges link firms that are often hacked by the same researcher. We note that
this representation of firms in a network only makes sense if researchers tend
to specialize on certain kinds of firms and repeatedly interact with the same
ones. This visualization provides an overview of the how different firms are
grouped together into sub-markets.

To construct this network of firms we consider all vulnerabilities reported in
2020. For the sake of visualization, we remove firms with less than 10
reported vulnerabilities, and do not consider researchers submitting only one
report. We connect two firms with an edge weighted by the number of
researchers they have in common. We then filter the network for statistically
significant co-occurrences using the Noise-Corrected Backbone method of Coscia
and Neffke (2017) (with the NC parameter set to 35). This method compares the
observed distribution of edge weights against a hypergeometric statistical
null model in which researchers and firms have the same number of
interactions, but are otherwise randomized. The remaining edges represent
pairs of firms which have highly significant overlap in their researchers. We
visualize the network in Figure 2, dropping firms that have become isolated
after the edge filtering process.

Figure 2: The network of firms on HackerOne using data from 2020. Two firms
are connected if the same researchers report vulnerabilities to both. Edges
are filtered for statistically significant co-occurrences of researchers
(using the Noise-Correct Backbone method of Coscia and Neffke (2017). The
network is plotted with labels in A), while the inset B) is the same network
without labels. To some extent we observe clustering of entities in related
sectors - commercial internet platforms, pornographic websites, open source
software communities, and cryptocurrency related firms.

This qualitative visualization of the data serves several purposes. First, it
indicates the variety of firms involved in BBPs on HackerOne, and that firms
or entities in similar sectors are close in this network. For instance,
pornographic websites are close to one another; as are open source software
supporting entities and firms in the cryptocurrency space. It also suggests
that there are significant latent similarities between certain firms that
researchers are exploiting, likely relating to the technologies and systems
they use. Future research could investigate how researchers navigate this
landscape of firms.

## Discussion

In this paper we examined the organization of bug bounty programs,
highlighting first the significant transaction costs arising from information
asymmetry problems, and then studying how centralized online platforms like
HackerOne reduce these problems and their costs. Doing so, these platforms
make a better market for information security. They do so by enabling their
participants to better signal information about themselves and to capture the
value of their activity. Our work suggests that central platforms play a
crucial role in the market for BBPs (Zhao et al., 2017). Our work fits into a
broader research program that demonstrates the continued relevance of the
transaction cost perspective in the context of new digital economy sectors
like the sharing economy and market platforms (Henten and Windekilde, 2016;
Nagle et al., 2020). Our findings highlight the unique incentives and costs
that researchers and firms face in this market.

We observed that researchers are more likely to link their HackerOne profiles
to their other identities on the web as they are more active on the site. This
observed correlation is even stronger among researchers with disclosed
vulnerability reports in their records. These reports provide significant
detail on the vulnerability, and act as convincing proof of a researcher’s
ability, akin to an example in a designer’s portfolio.

On the firm side, we found that firms are significantly more likely to
disclose vulnerabilities early on in their tenure on HackerOne. We interpreted
this as evidence that firms wish to signal their collaborative quality to the
wider crowd. Once that track record has been established, firms seem to become
more conservative with what they release.

Finally we observed that over the history of HackerOne, researchers tended to
become more specialized, interacting more frequently with the same firms. We
interpreted this as a sign of the maturity of the market in which embedded
relationships are increasingly important. We also considered an explanation
involving information effects: as the history of firm activity on HackerOne
grows, new researchers can more easily match with compatible firms because
they can observe past firm behavior.

Despite the proliferation of BBPs, no doubt in part because of the success of
platforms like Hackerone, there are still significant problems with the way
BBPs work in practice (Ellis and Stevens, 2022; Goerzen and Coleman, 2022).
For example, payment for work on HackerOne can be highly variable. A
researcher can work on a vulnerability report, only to find that another
researcher had already pointed out that bug. Usually these researchers get no
financial rewards. For many researchers it may be difficult to generate a
reliable source of income from such work. While the potential for learning and
the labor market value of a strong HackerOne track record compensate
researchers to some extent, such issues likely limit participation. In this
way BBPs suffer many of the same problems that other gig economy jobs do
(Graham et al., 2017). Future work may consider how to optimize the design and
incentives of BBPs with a view to expanding participation or smoothing out
inequalities (Bokányi and Hannák, 2020) on the researcher side.

The digital traces left by researchers in BBPs likely contain valuable
information for the information security community. To date, we are not aware
of efforts to mine data insights from disclosed vulnerabilities on platforms
such as HackerOne. Such data could present a crowd intelligence complement to
other sources of information such as vulnerability reports published by
research institutes and governments. These mainstream sources are already
collected and combined in a structured format, for instance in the SEPSES
Knowledge Graph (Kiesling et al., 2019). Such databases form the potential
basis of machine learning-based countermeasures or warning systems for
vulnerabilities (Eckhart et al., 2020). More generally, the question of how
BBPs work with or complement traditional methods of security auditing remains
largely unanswered.

Another promising avenue of questions about BBPs relates to entry into this
market on the firm side. Information security is a high risk and sensitive
area for most firms and institutions. Internal actors may rather ignore
potential problems than acknowledge them, a fact reflected in widespread
underinvestment in information security among private sector firms (Gordon et
al., 2018). Some firms may not feel comfortable engaging with semi-anonymous
researchers. Many potential market participants may feel blocked from
participation because they are uncomfortable with variability in the
expenditures on security. This latter point may be one reason why public
sector institutions are underrepresented in this space, as they are in other
new economy sectors such as the sharing economy (Ganapati and Reddick, 2018).
This last point raises two practical questions. First, why are institutions
hesitant to try BBPs, and to what extent is it because of internal
organization or culture? Second, can BBPs be designed, perhaps via platforms
like HackerOne, in a way that overcomes these obstacles? These questions are
essential ones if we want to expand participation in BBPs, and to enjoy the
resulting potential improvements in digital security.

## Acknowledgements

I would like to acknowledge and thank Sebastian Bockholt, Lukas Ochse, Edgar
Weippl, Anna Weichselbraun, Sandor Juhasz, William Schueller, Gergo Toth, and
Aleksandra Urman for helpful conversations that improved this manuscript.

## References

  * Agrafiotis et al. (2018) Agrafiotis I, Nurse JR, Goldsmith M, Creese S, Upton D (2018) A taxonomy of cyber-harms: Defining the impacts of cyber-attacks and understanding how they propagate. Journal of Cybersecurity 4(1):tyy006
  * Agrawal et al. (2014) Agrawal A, Catalini C, Goldfarb A (2014) Some simple economics of crowdfunding. Innovation policy and the economy 14(1):63–97
  * Ahmed (2021) Ahmed EM (2021) Modelling information and communications technology cyber security externalities spillover effects on sustainable economic growth. Journal of the Knowledge Economy 12(1):412–430
  * Alt (2017) Alt R (2017) Electronic markets on transaction costs
  * Alt and Klein (2011) Alt R, Klein S (2011) Twenty years of electronic markets research—looking backwards towards the future. Electronic Markets 21(1):41–51
  * Alt and Zimmermann (2019) Alt R, Zimmermann HD (2019) Electronic markets on platform competition
  * Anderson (2001) Anderson R (2001) Why information security is hard-an economic perspective. In: Seventeenth Annual Computer Security Applications Conference, IEEE, pp 358–365
  * Biro et al. (2017) Biro M, Mashkoor A, Sametinger J, Seker R (2017) Software safety and security risk mitigation in cyber-physical systems. IEEE Software 35(1):24–29
  * Blind (2016) Blind K (2016) The impact of standardisation and standards on innovation. In: Handbook of innovation policy impact, Edward Elgar Publishing
  * Bokányi and Hannák (2020) Bokányi E, Hannák A (2020) Understanding inequalities in ride-hailing services through simulations. Scientific reports 10(1):1–11
  * Braesemann et al. (2021) Braesemann F, Stephany F, Teutloff O, Kässi O, Graham M, Lehdonvirta V (2021) The polarisation of remote work. Available at SSRN
  * Buchanan (2020) Buchanan B (2020) The Hacker and the State: Cyber Attacks and the New Normal of Geopolitics. Harvard Uni. Press
  * Capon et al. (1988) Capon N, Hulbert JM, Farley JU, Martin LE (1988) Corporate diversity and economic performance: The impact of market specialization. Strategic Management Journal 9(1):61–74
  * Caudill et al. (1988) Caudill SB, et al. (1988) An advantage of the linear probability model over probit or logit. Oxford Bulletin of Economics and Statistics 50(4):425–427
  * Coscia and Neffke (2017) Coscia M, Neffke FM (2017) Network backboning with noisy data. In: 2017 IEEE 33rd International Conference on Data Engineering (ICDE), IEEE, pp 425–436
  * Di Maggio et al. (2017) Di Maggio M, Kermani A, Song Z (2017) The value of trading relations in turbulent times. Journal of Financial Economics 124(2):266–284
  * Dupont and Lusthaus (2021) Dupont B, Lusthaus J (2021) Countering distrust in illicit online networks: The dispute resolution strategies of cybercriminals. Social Science Computer Review p 0894439321994623
  * Duxbury and Haynie (2021) Duxbury SW, Haynie DL (2021) Network embeddedness in illegal online markets: endogenous sources of prices and profit in anonymous criminal drug trade. Socio-Economic Review
  * Eckhart et al. (2020) Eckhart M, Ekelhart A, Weippl ER (2020) Automated security risk identification using automationml-based engineering data. IEEE Transactions on Dependable and Secure Computing
  * Ellis and Stevens (2022) Ellis R, Stevens Y (2022) Bounty everything: Hackers and the making of the global bug marketplace. Data and Society
  * Ganapati and Reddick (2018) Ganapati S, Reddick CG (2018) Prospects and challenges of sharing economy for the public sector. Government Information Quarterly 35(1):77–87
  * Goel and Shawky (2009) Goel S, Shawky HA (2009) Estimating the market impact of security breach announcements on firm values. Information & Management 46(7):404–410
  * Goerzen and Coleman (2022) Goerzen M, Coleman G (2022) Wearing many hats. Data and Society
  * Gordon et al. (2014) Gordon LA, Loeb MP, Lucyshyn W, Zhou L, et al. (2014) Externalities and the magnitude of cyber security underinvestment by private sector firms: a modification of the gordon-loeb model. Journal of Information Security 6(01):24
  * Gordon et al. (2015) Gordon LA, Loeb MP, Lucyshyn W, Zhou L (2015) The impact of information sharing on cybersecurity underinvestment: A real options perspective. Journal of Accounting and Public Policy 34(5):509–519
  * Gordon et al. (2018) Gordon LA, Loeb MP, Lucyshyn W, Zhou L, et al. (2018) Empirical evidence on the determinants of cybersecurity investments in private sector firms. Journal of Information Security 9(02):133
  * Graham et al. (2017) Graham M, Hjorth I, Lehdonvirta V (2017) Digital labour and development: impacts of global digital labour platforms and the gig economy on worker livelihoods. Transfer: European review of labour and research 23(2):135–162
  * HackerOne (2019) HackerOne (2019) The 2019 hacker report. Tech. rep.
  * HackerOne (2021) HackerOne (2021) The 2021 hacker report. Tech. rep.
  * Hargittai (2021) Hargittai E (2021) Handbook of digital inequality. Edward Elgar Publishing
  * Hata et al. (2017) Hata H, Guo M, Babar MA (2017) Understanding the heterogeneity of contributors in bug bounty programs. In: 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), IEEE, pp 223–228
  * Henten and Windekilde (2016) Henten AH, Windekilde IM (2016) Transaction costs and the sharing economy. info
  * Hernandez et al. (2018) Hernandez L, Vignes A, Saba S (2018) Trust or robustness? an ecological approach to the study of auction and bilateral markets. PloS one 13(5):e0196206
  * Kiesling et al. (2019) Kiesling E, Ekelhart A, Kurniawan K, Ekaputra F (2019) The sepses knowledge graph: an integrated resource for cybersecurity. In: International Semantic Web Conference, Springer, pp 198–214
  * Laszka et al. (2018) Laszka A, Zhao M, Malbari A, Grossklags J (2018) The rules of engagement for bug bounty programs. In: International Conference on Financial Cryptography and Data Security, Springer, pp 138–159
  * LaToza and Van Der Hoek (2015) LaToza TD, Van Der Hoek A (2015) Crowdsourcing in software engineering: Models, motivations, and challenges. IEEE software 33(1):74–80
  * Libicki et al. (2014) Libicki MC, Senty D, Pollak J (2014) Hackers wanted: An examination of the cybersecurity labor market. Rand Corporation
  * Maillart et al. (2017) Maillart T, Zhao M, Grossklags J, Chuang J (2017) Given enough eyeballs, all bugs are shallow? revisiting eric raymond with bug bounty programs. Journal of Cybersecurity 3(2):81–90
  * Malladi and Subramanian (2019) Malladi SS, Subramanian HC (2019) Bug bounty programs for cybersecurity: Practices, issues, and recommendations. IEEE Software 37(1):31–39
  * Malone et al. (1987) Malone TW, Yates J, Benjamin RI (1987) Electronic markets and electronic hierarchies. Communications of the ACM 30(6):484–497
  * Marlow and Dabbish (2013) Marlow J, Dabbish L (2013) Activity traces and signals in software developer recruitment and hiring. In: Proceedings of the 2013 conference on Computer supported cooperative work, pp 145–156
  * Mumtaz et al. (2020) Mumtaz S, Rodriguez C, Zamanirad S (2020) Security professional skills representation in bug bounty programs and processes. In: International Conference on Service-Oriented Computing, Springer, pp 334–348
  * Nagle et al. (2020) Nagle F, Seamans R, Tadelis S (2020) Transaction cost economics in the digital economy: A research agenda. Harvard Business School Strategy Unit Working Paper (21-009)
  * Nedelkoska et al. (2019) Nedelkoska L, Neffke F, et al. (2019) Skill mismatch and skill transferability: Review of concepts and measurements. Papers in Evolutionary Economic Geography
  * Neffke (2019) Neffke FM (2019) The value of complementary co-workers. Science advances 5(12):eaax3370
  * Papoutsoglou et al. (2021) Papoutsoglou M, Wachs J, Kapitsaki GM (2021) Mining dev for social and technical insights about software development. In: 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), IEEE, pp 415–419
  * Paquet-Clouston et al. (2018) Paquet-Clouston M, Décary-Hétu D, Morselli C (2018) Assessing market competition and vendors’ size and scope on alphabay. International Journal of Drug Policy 54:87–98
  * Pavlou and Gefen (2004) Pavlou PA, Gefen D (2004) Building effective online marketplaces with institution-based trust. Information systems research 15(1):37–59
  * Powell (1990) Powell WW (1990) Neither market nor hierarchy: Network forms of organization. Research in Organizational Behavior 12:295–336
  * Raymond (1999) Raymond E (1999) The cathedral and the bazaar. Knowledge, Technology & Policy 12(3):23–49
  * Riedl et al. (2021) Riedl C, Kim YJ, Gupta P, Malone TW, Woolley AW (2021) Quantifying collective intelligence in human groups. Proceedings of the National Academy of Sciences 118(21)
  * Svangren et al. (2017) Svangren MK, Skov MB, Kjeldskov J (2017) The connected car: an empirical study of electric cars as mobile digital devices. In: Proceedings of the 19th international conference on human-computer interaction with mobile devices and services, pp 1–12
  * Uzzi (1996) Uzzi B (1996) The sources and consequences of embeddedness for the economic performance of organizations: The network effect. American sociological review pp 674–698
  * Wachs and Vedres (2021) Wachs J, Vedres B (2021) Does crowdfunding really foster innovation? evidence from the board game industry. Technological Forecasting and Social Change 168:120747
  * Wachs et al. (2022) Wachs J, Nitecki M, Schueller W, Polleres A (2022) The geography of open source software: Evidence from github. Technological Forecasting and Social Change 176:121478
  * Williamson (1979) Williamson OE (1979) Transaction-cost economics: the governance of contractual relations. The journal of Law and Economics 22(2):233–261
  * Williamson (1991) Williamson OE (1991) Comparative economic organization: The analysis of discrete structural alternatives. Administrative science quarterly pp 269–296
  * Zhao and Zhao (2010) Zhao JJ, Zhao SY (2010) Opportunities and threats: A security assessment of state e-government websites. Government Information Quarterly 27(1):49–56
  * Zhao et al. (2017) Zhao M, Laszka A, Grossklags J (2017) Devising effective policies for bug-bounty platforms and security vulnerability discovery. Journal of Information Policy 7:372–418