# Sandboxing Adoption in Open Source Ecosystems

Maysara Alhindi  University of BristolUnited Kingdom
<EMAIL_ADDRESS>and  Joseph Hallett  University of BristolUnited
Kingdom<EMAIL_ADDRESS>
(2024)

###### Abstract.

Sandboxing mechanisms allow developers to limit how much access applications
have to resources, following the least-privilege principle. However, it’s not
clear how much and in what ways developers are using these mechanisms. This
study looks at the use of _Seccomp_ , _Landlock_ , _Capsicum_ , _Pledge_ , and
_Unveil_ in all packages of four open-source operating systems. We found that
less than 1% of packages directly use these mechanisms, but many more
indirectly use them. Examining how developers apply these mechanisms reveals
interesting usage patterns, such as cases where developers simplify their
sandbox implementation. It also highlights challenges that may be hindering
the widespread adoption of sandboxing mechanisms.

††journalyear: 2024††copyright: rightsretained††conference: 2024 12th ACM/IEEE
International Workshop on Software Engineering for Systems-of-Systems and
Software Ecosystems; April 14, 2024; Lisbon, Portugal††booktitle: 2024 12th
ACM/IEEE International Workshop on Software Engineering for Systems-of-Systems
and Software Ecosystems (SESoS ’24 ), April 14, 2024, Lisbon, Portugal††doi:
10.1145/3643655.3643878††isbn: 979-8-4007-0557-1/24/04

##  1\. Introduction

You can’t always trust that the software you use will behave as expected.
Attackers may exploit privileged software to increase their level of access.
Sandboxes help prevent this by following the _principle of the least
privilege_ (Saltzer and Schroeder, 1975), ensuring that programs run with the
minimum functionality needed. If a program shouldn’t access specific files or
network devices, the sandbox prevents it, safeguarding against potential
exploits.

In open-source ecosystems, sandboxes play a crucial role as systems are
constructed by combining software into _packages_ to meet specific deployment
requirements. While most contemporary operating systems offer some type of
sandboxing mechanisms, unlike many OS features, sandboxing functionality
(aside from basic chroots and the Unix DAC) lacks standardization. Each
operating system presents unique mechanisms, at times featuring multiple
competing options. Consequently, if software intends to leverage sandboxing,
it needs customization to align with each OS it aims to support.

To ensure software is properly tailored for an operating system, open source
OS vendors provide package management tools and repositories. In these
repositories, software is adjusted to fit within the OS, disabling unsupported
features, aligning installation paths, and applying patches for OS-specific
functionality.

Various operating systems offer different sandboxing mechanisms. Linux employs
_Seccomp_ and the Linux Security Module (LSM) framework, allowing for
pluggable kernel modules like _Landlock_. FreeBSD has _Capsicum_ for
sandboxing, while OpenBSD enables developers to sandbox applications using
_Pledge_ and _Unveil_ by specifying system calls and file paths.

To what degree do software packages in various open-source ecosystems employ
sandboxing mechanisms? Are certain operating systems offering a greater
proportion of sandboxed software than others, and how do the functionalities
of these sandboxes differ? In pursuit of answers, we examined source package
archives for four open source OSs (Table 1) to identify packages that utilize
sandboxing mechanisms.

Table 1. Different OSs examined and their packages. OS | Version | Packages
---|---|---
Debian Linux | Bullseye 11 | 59,913
Fedora Linux | 37 | 66,166
OpenBSD | 7.3 | 7,787
FreeBSD | 13.2 | 30,766

Specifically, this paper addresses the following research questions:

  * •

RQ1: To what extent sandboxing mechanisms are used in operating systems
packages?

  * •

RQ2: How do sandboxing mechanisms compare in terms of complexity and security
when applied to the same package in different operating systems?

We observed that although many packages rely on packages that employ
sandboxing mechanisms, the actual number of packages directly using sandboxing
APIs appears relatively small across all operating systems. When examining the
packages that utilize these mechanisms, we found instances where developers
didn’t fully use the capabilities of fine-grained sandboxing mechanisms, such
as those in Seccomp. Instead, they tended to simplify their approaches,
mapping it to simpler models like Pledge’s.

Additionally, we found that using sandboxing mechanisms requires developers to
restructure their code to align with sandboxing requirements. Interestingly,
Pledge’s model of sandboxing sometimes offers features that result in a more
restrictive sandbox compared to Seccomp and Capsicum.

##  2\. Background

###  2.1. Sandboxing mechanisms

####  2.1.1. Seccomp

_Seccomp_ was introduced in Linux in 2005 (short for secure computing),
offering developers an API to limit the system calls their applications can
make. It is inspired by the now-defunct _Systrace_ (Kurchuk and Keromytis,
2004). With _Seccomp_ , developers can create allow or deny list of system
calls and configure eight different actions for filtered calls. For example,
_Seccomp_ can trap or kill a process upon execution of a system call,
preventing attackers from relaunching their attacks. _Seccomp_ Filters are
inherited by child processes/threads.

Developers using _Seccomp_ can write Berkeley Packet Filter (BPF) code to
specify allowed system calls and their arguments. BPF code, resembling
assembly language, runs in the kernel and has restrictions, such as a maximum
size limit and a limited set of instructions (Calavera and Fontana, 2019). The
BPF interface enables examination of system call arguments, but it can’t
dereference pointer arguments. This limitation is to avoid Time-of-Check to
Time-of-Use (TOCTOU) attacks (Canella et al., 2021). For instance, while you
can create BPF filters allowing the read system call with specific file
descriptors, filtering the fopen system call and checking its string path
argument isn’t possible since the argument is a pointer.

_Seccomp_ provides a C library (libseccomp) that enables filtering system
calls without the need to write BPF code directly. Listing 1 shows an example
of _Seccomp_ used with libseccomp, restricting a process to solely execute the
write system call when the passed argument corresponds to the stdout file
descriptor.

⬇

scmp_filter_ctx ctx;

ctx = seccomp_init(SCMP_ACT_KILL);

seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(write), 1, SCMP_A0(SCMP_CMP_EQ,
STDOUT_FILENO));

seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(exit_group), 0);

seccomp_load(ctx);

Listing 1: Seccomp filter that allows the write system call to stdout file
descriptor

####  2.1.2. Landlock

_Landlock_ was introduced as a Linux security module in 2017 (Salaün, 2017).
It empowers developers to associate access rights rules with files and
directories, enabling, for example, restrictions like read-only access to the
filesystem. _Landlock_ offers four access rights for files and eleven for
directories. While it allows additional filters to restrict process access, it
doesn’t permit their expansion. Notably, files or directories opened before
creating the sandbox remain unrestricted, and similar to _Seccomp_ , child
processes inherit the sandbox from their parent. The example in Listing 2
demonstrates _Landlock_ usage to confine process access to read-only
permissions on the /usr directory, extracted and simplified from _Landlock_
man pages.

⬇

landlock_create_ruleset(&attr, sizeof(attr), 0);

struct landlock_path_beneath_attr path_beneath = {0};

path_beneath.allowed_access = LANDLOCK_ACCESS_FS_EXECUTE | LANDLOCK_ACCESS_FS_READ_FILE | LANDLOCK_ACCESS_FS_READ_DIR;

path_beneath.parent_fd = open("/usr", O_PATH | O_CLOEXEC);

landlock_add_rule(ruleset_fd, LANDLOCK_RULE_PATH_BENEATH, &path_beneath, 0);

close(path_beneath.parent_fd);

prctl(PR_SET_NO_NEW_PRIVS,1, 0, 0, 0)

landlock_restrict_self(ruleset_fd, 0)

Listing 2: Landlock used to only allow read access to /usr folder

####  2.1.3. Pledge and Unveil

OpenBSD provides _Pledge_ and _Unveil_ system calls, which are similar to
_Seccomp_ and _Landlock_ but have a simpler interface and user experience
(Beck, 2019). Previously known as _Tame_ , _Pledge_ allows developers to
declare that their program will only use specific promises. Promises are
collections of privileges and system calls with similar functionality,
defining the scope of permissible actions. For example, the unix promise
groups 10 system calls related to operating on the AF_UNIX domain, while the
id promise encompasses system calls related to process rights and identity
(OpenBSD documentation, 2022).

_Unveil_ limits applications to only use specified directories with designated
permissions, though not as finely detailed as those provided by _Landlock_.
Developers can employ _Unveil_ with permissions like r, w, x, c, representing
read, write, execute, and create/remove permissions for the path (OpenBSD
documentation, 2021).

Any attempt to use permissions outside the pledged promises will result in the
program being terminated, unless the error promise is used, in which case an
error is returned. Unlike _Seccomp_ , child processes don’t inherit the
sandbox restrictions.

⬇

pledge("stdio␣tty", NULL);

unveil("/folder", "r");

Listing 3: Example of Pledge and Unveil

####  2.1.4. Capsicum

In FreeBSD, _Capsicum_ serves as a sandboxing mechanism designed to help
developers restrict their applications’ resource access (Zaborski, 2016).
_Capsicum_ equips developers with various system calls to control their
application’s access to file descriptors, covering files, sockets, and other
entities represented as file descriptors in the kernel (Watson et al., 2010).
Once in the capability mode, achieved through `cap_enter`, the process is
barred from accessing the global namespace, including reading from the
filesystem. Additionally, _Capsicum_ enables the assignment of capability
rights and permissions to file descriptors. For example, this can be used to
permit only reading access to a specific file. Moreover, _Capsicum_
facilitates the management of allowed IOCTLs and FCNTLs numbers for an
application. Accessing resources beyond the allowed scope doesn’t terminate
the program but triggers a capability violation error. Similar to _Seccomp_ ,
child processes of a capability-mode process will also be sandboxed.

The following example, extracted from the FreeBSD _Capsicum_ man pages,
illustrates how _Capsicum_ can be employed to restrict a file descriptor to
read-only permission.

⬇

cap_rights_t setrights;

char buf[1];

int fd;

fd = open("/tmp/foo", O_RDWR);

if (fd < 0)

err(1, "open()␣failed");

if (cap_enter() < 0)

err(1, "cap_enter()␣failed");

cap_rights_init(&setrights, CAP_READ);

if (cap_rights_limit(fd, &setrights) < 0)

err(1, "cap_rights_limit()␣␣failed");

buf[0] = ’X’;

if (write(fd, buf,sizeof(buf)) > 0)

errx(1, "write()␣succeeded!");

if (read(fd, buf, sizeof(buf)) < 0)

err(1, "read()␣failed");

Listing 4: Capsicum code that restricts a file descriptor to read-only mode

##  3\. Method

To assess the adoption of sandboxing mechanisms in software across different
ecosystems, we downloaded the source code for all packages within the x86-64
Debian and Fedora repositories, as well as the ports trees of OpenBSD and
FreeBSD. This data collection and analysis took place from April to August
2023. We specifically chose these operating systems because they represent two
major Linux distributions, forming the foundation for numerous other
distributions that build upon their software. Additionally, FreeBSD and
OpenBSD, though less prevalent, offer distinctive sandboxing mechanisms and
cater to specific niches within the broader open source OS ecosystem. OpenBSD,
in particular, is known for prioritizing security and actively implementing
sandboxing measures for applications, making it a noteworthy focus for
investigation.

To identify packages that use sandboxing mechanisms, we used regular
expressions to search code that invokes sandbox mechanisms system calls.
Specifically, we searched within .c, .cpp, .cc, .h, .hpp, and .sh files for
calls to:

  * •

seccomp_init

  * •

landlock_restrict_self

  * •

pledge

  * •

unveil

  * •

cap_enter

  * •

cap_rights_limits

  * •

cap_ioctl_limits

  * •

cap_fcntls_limit

For _Seccomp_ and _Landlock_ , we also searched for invocations of prctl and
syscall with the appropriate arguments.

We took measures to minimize false positives, such as filtering out noise like
system call invocations in commented code, documentation, or strings. We also
filtered out instances where sandboxing mechanisms are employed solely in test
cases rather than for creating a sandbox. We observed a number of compiler
packages containing _llvm_ source code where _Seccomp_ filters are installed
only to test whether ASAN (Address Sanitizer) works properly under sandbox
restrictions. Similarly, various kernel-related packages include tests to
ensure that _Seccomp_ and _Landlock_ function correctly. However, none of
these packages utilize sandboxing mechanisms to establish a sandbox; the
mechanisms are used only in testing code. We also manually checked packages
that passed these filters to make sure that we didn’t include any false
positives.

Additionally, our analysis accounts for instances where multiple packages
share the same source code. For example, packages like _firefox-esr-l10n-en-
gb_ and _firefox-esr-l10n-en-ca_ in Debian share identical source code despite
generating different packages (distinguished by language). Our analysis
considers results from all packages, even if they originate from the same
source code.

We also compared how different operating systems make use of their sandboxes.
We focused on packages existing in different operating systems that utilize at
least two different sandboxing mechanisms in their codebases. Our selection
aimed to highlight interesting usage patterns and compare how developers
implement different sandboxing mechanisms for the same package across
different operating systems. This delves into the security implications and
complexity associated with implementing sandboxes.

##  4\. Threats to validity

There are several threats to validity of this study, the largest of which
relates to how we identify sandboxed packages. We focus our analysis on C and
C++ source code since not all sandboxing mechanisms have official libraries
for the other languages, but all make system calls that are exposed through C
and C++ bindings. To help account for this when looking at whether a package
makes use of sandboxing features we first look for packages directly calling
the APIs, and also for packages declaring a dependency on packages which make
use of a sandboxing API. This means that if a package provides native bindings
for a sandbox API (such as ruby-pledge in OpenBSD which provides Ruby bindings
for the pledge syscall) then we capture programs that use sandboxing features
through the API but without making C or C++ syscalls directly. It also means
that if a program depends on a sandboxed binary for some tiny aspect of its
functionality then we treat the whole package as being sandboxed, as in pssh,
ssvnc, vagrant and many other packages that depend on openssh-client package
which is sandboxed using Seccomp. Whilst in truth it may be only some minor
aspect that is palmed off on an external library. We favor including these
packages in our analysis as it’s not our place to determine to what extent a
package is sandboxed or how appropriately; but it is a weakness of the study.

Sometimes the source code of a package is unavailable due to unmaintained
websites or other problems. We failed to download the source code of less than
4% of the packages for any OS and didn’t include these packages in our study.
Additionally, for the BSDs core system software (such as the default shell) is
distributed as part of the OS release and not as part of an individual
package. This software is also omitted.

Another weakness is that whilst packages are _predominantly_ used for
packaging software, sometimes they’re not; some packages just add
documentation, for example. Our analysis doesn’t exclude non-software packages
since it is challenging to identify them.

##  5\. Results

###  5.1. Sandboxing adoption

Table 2. Percentage of packages that use sandboxing mechanisms. OS | Packages count | Sandboxed | Depend on Sandboxed
---|---|---|---
Debian | 59893 | 406 (0.68%) | 10164 (16.97%)
Fedora | 63706 | 396 (0.62%) | 23747 (37.28%)
OpenBSD | 7700 | 114 (1.48%) | 1903 (24.71%)
FreeBSD | 30763 | 43 (0.14%) | 13900 (45.18%)
Table 3. Number of packages in each OS with sandboxing mechanisms, per sandboxing tool. OS | Seccomp | Pledge | Unveil | Capsicum
---|---|---|---|---
Debian | 406 | x | x | x
Fedora | 396 | x | x | x
OpenBSD | x | 111 111There are 3 packages that use Unveil without Pledge. | 36 | x
FreeBSD | x | x | x | 43

Table 2 shows the percentage of packages that use sandboxing mechanisms in the
studied OSs and table 3 shows usage statistics for each of the sandboxing
mechanisms. Whilst each OS contains thousands of packages, only a fraction of
them (for all OSs ¡1%) ever seem to use the sandbox APIs directly. Instead,
packages seem to _depend_ on other packages (that is, required and installed
first) that either use or again depend on sandboxed code.

_Landlock_ is only used in kernel packages in testing code and a sample
program, but nothing aside from that, it’s not used in any other packages to
create a sandbox.

###  5.2. What kinds of apps get sandboxed?

We categorized packages that directly use sandboxing mechanisms, each
operating system has its own category system but to allow comparison between
OSs we used Debian’s label for other OSs. Table 4 shows the 10 most popular
categories in packages that use sandboxing mechanisms in all operating
systems.

For Linux, the majority of the packages that make use of sandboxed code are
for development and libraries (Table 4), but for OpenBSD and FreeBSD it is
instead the networking and utility applications that mostly make use of
sandboxing mechanisms. Looking at results without context can be misleading,
for instance, Debian has 104 browser packages, this is because 97 of them are
different versions of _firefox_ , while Fedora has 4, and FreeBSD and OpenBSD
have 2 packages for _firefox_. When taking a closer look at the development
packages in both Fedora and Debian, we notice that they are mostly packages
for popular hypervisors like _zen_ and _qemu_ and containers solutions like
_lxc_ , there are also packages for fuzzing and testing tools like _rumur_ and
_afl_. In OpenBSD, the development category contains a more diverse set of
software such as development frameworks _arcan_ , web applications _sblg_ ,
_stagit_ and development tools such as _openradtool_ and
_the_silver_searcher_.

Network tools are prime candidates for sandboxing as they are typically
exposed to external users. The BSDs have a higher percentage of packages under
the networking category that employ a sandboxing mechanism. Most of the
networking packages in Debian and Fedora are different versions of _ntp_ ,
_openssh_ and _gnutls_ , for instance, Fedora has 10 _gnutls_ packages and
Debian has 5 _openssh_ packages. The BSDs, especially OpenBSD has a broader
and more diverse software under the networking category, such as IRC clients
_catgirl_ , Gemini _gmid_ and Gopher _geomyidae_ servers, and VPN software
like _mlvpn_.

Overall there is a diverse range of apps that make use of sandboxing
mechanisms from document readers, administrator tools, and graphics tools to
games, such as _warzone2100_ that uses _Pledge_ , and _redeclipse_ that uses
_Seccomp_. Mostly, however, the mechanisms are used in networking and
development packages.

Table 4. Categories of packages that directly use sandboxing mechanisms. The table includes the 10 most used categories across all OSs. OS | devel | libs | net | browser | utils | admin | mail | javascript | misc | python
---|---|---|---|---|---|---|---|---|---|---
Debian | 34 (8%) | 105 (26%) | 28 (7%) | 104 (26%) | 11 (3%) | 36 (9%) | 67 (17%) | 1 (0%) | 10 (2%) | 3 (1%)
Fedora | 141 (36%) | 62 (16%) | 41 (10%) | 13 (3%) | 40 (10%) | 55 (14%) | 3 (1%) | 28 (7%) | 7 (2%) | 4 (1%)
OpenBSD | 11 (10%) | 2 (2%) | 38 (33%) | 3 (3%) | 40 (35%) | 4 (4%) | 4 (4%) | 1 (1%) | 1 (1%) | 0
FreeBSD | 2 (5%) | 3 (7%) | 12 (28%) | 3 (7%) | 8 (19%) | 5 (12%) | 2 (5%) | 2 (5%) | 0 | 1 (2%)

###  5.3. What is the sandboxed code doing?

####  5.3.1. Linux

With _Seccomp_ , developers can examine system calls’ arguments and limit
their program to run system calls only if certain arguments are provided. For
instance, you can limit programs to only write to stdout or limit the ioclt
system call to be run only with specific numbers. To do that, developers have
to write BPF filters. BPF filters have a syntax similar to assembly and can be
challenging to write. One would assume that developers would tend to simplify
their _Seccomp_ sandbox and only use it to allow/deny system calls without
going the extra mile and write BPF code to limit their arguments. Our analysis
shows the opposite. We manually checked packages that use _Seccomp_ and found
that 79% of them in Fedora and 82% in Debian utilize it to check system calls’
arguments and further restrict their sandbox. When checking for packages that
use libseccomp, we found that 40% of packages in Debian and 76% in Fedora use
libseccomp and the rest write BPF code to implement the _Seccomp_ sandbox.

####  5.3.2. OpenBSD

We collected promises that are used with every call to _Pledge_. The stdio
promise is the most used promise, it is used in almost every call to _Pledge_.
The stdio promise grants access to 69 system calls that are mostly related to
working with standard input and output and are essential for libc to work.
Followed by rpath which allows system calls that have a read-only effect,
wpath that allows system calls that have a write-only effect, and cpath that
allows system calls that create new files or directories (OpenBSD
documentation, 2022). Software, in general, uses the file system in one way or
another, and these results suggest that a sandboxing mechanism that offers the
ability to restrict the file system actions will be appealing to and used by
developers, in contrast to some promises that are never used in any of the
packages in OpenBSD such as mcast that allows operating on multicast sockets,
tape which allows certain IOCTLs on tape drivers, settime that allows setting
the system time and error which makes _Pledge_ returns an error instead of
killing the program upon a violation of the pledged promises.

####  5.3.3. FreeBSD

_Capsicum_ is used by 43 packages in the FreeBSD ports tree. Capsicum provides
different system calls to enable sandboxing, the cap_enter system call that
puts the process in capability mode restricting access to the global
namespaces is used in 34 packages, while cap_rights_limit is used 27 times,
this system call is used to impose restrictions on file descriptors. The
cap_ioctls_limit system call controls the access to IOCTLs and is used 7
times. The cap_fcntls_limit that manages access to fcntl commands is used 6
times.

###  5.4. What are they sandboxing?

Are sandboxes for similar packages equivalent between OSs? We searched the
database for packages that exist in different operating systems and use their
sandboxing mechanisms, then we examined how different sandboxes were applied
to secure the packages across different operating systems. We were interested
in observing how developers utilize the sandboxing mechanisms when used to
secure the same package in terms of the security and complexity of the
implemented sandbox. Whilst this isn’t an exhaustive comparison, we can survey
how some packages differ when run on different OSs.

Having a fine-grained sandboxing mechanism like _Seccomp_ enables developers
to create detailed sandboxes that only allow what an application needs and
nothing more, in contrast to sandboxing mechanisms that don’t allow that level
of detailed sandboxing like _Pledge_. This was the case for a number of
applications that we examined, like _arping_ package that exists in Linux,
OpenBSD, and FreeBSD. It uses _Pledge_ with only two promises, stdio, and tty.
The stdio promise allows 69 system calls that allow the program to perform
basic I/O operations using the standard input and output, and it allows the
functionality of libc to work. The tty promise grants the application read and
write permissions on a /dev/tty device, and it grants permissions to perform
11 IOCTL operations that are related to controlling and interacting with the
terminal device. The package uses _Unveil_ to grant access to all of the root
filesystem. However, _arping_ package implements a stricter and more secure
sandbox using _Seccomp_. It limits the process access to only 9 system calls,
fstat, write, ioctl, sendto, select, pselect6, newfstatat, exit_group and
rt_sigreturn. In addition, the sandbox utilizes BPF filters to limit fstat and
write system calls to use only standard output and error and to limit the
numbers passed to both the ioctl and sendto system calls. In FreeBSD, the
package doesn’t use any sandboxing mechanisms.

But fine-grained sandboxing doesn’t always seem to be used to create a
stricter and more detailed sandboxing. The _xwallpaper_ package exists in
Debian, FreeBSD and OpenBSD, and it uses _Pledge_ and _Seccomp_ to create
sandboxes. Contrary to _arping_ package, the sandbox created by _Seccomp_
isn’t more secure or restricted than the _Pledge_ sandbox. The package uses
_Pledge_ twice, at the beginning of the main function, _Pledge_ is used with 6
promises, dns, inet, proc rpath stdio and unix, and later, _Pledge_ is used to
drop privileges and only allow one promise, stdio. When using _Seccomp_ , the
package has two functions to implement the _Seccomp_ sandbox, stage1_sandbox()
and stage2_sandbox(). In both functions, the developer clearly attempted to
mimic the sandbox created by _Pledge_. The developer used _Seccomp_ to create
allow list of system calls that match exactly what _Pledge_ promises map to,
this is shown also in the code comments as shown below. In ‘stage2_sandbox’
function, the developer allow-listed the 69 system calls that are granted by
the stdio promise.

⬇

/* pledge: dns */

seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(connect), 0)

seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(sendto), 0)

seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(socket), 0)

...

/* pledge: inet+unix */

seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(accept), 0)

seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(accept4), 0)

...

Listing 5: Using Seccomp to mimic Pledge

When compared to _Pledge_ , _Seccomp_ and _Capsicum_ seem to be more flexible
and allow for detailed implementation of sandboxes. However, this isn’t always
the case, _Pledge_ has some features that allow developers to design more
secure sandboxes. This is apparent when used in the _gmid_ package, the
developer of this package wrote a blog about his experience when adopting the
different sandboxing mechanisms to _gmid_ package (Polo, 2021).

The _gmid_ package is a Gemini server that listens on a port, serves static
files, and runs CGI scripts. To accommodate the sandboxing mechanisms and to
implement a secure sandbox, the developer had to split the application into
two processes, one that listens on the port and the other one to execute CGI
scripts on demand. This split is necessary because child processes inherit the
sandbox of the parent process in both _Seccomp_ and _Capsicum_ , and without
the split, it would not be possible to fork and run CGI scripts since they
would be running under the parent sandbox and might not execute correctly
since they might need more permissions than the parent has. By splitting the
program into two processes, the listener and the executor, it would be
possible to use _Seccomp_ and _Capsicum_ to sandbox the listener process but
not the executor process. However, with _Pledge_ and _Unveil_ , it was
possible to sandbox both processes since the sandbox isn’t inherited by the
child process when forked to execute CGI scripts. Despite _Pledge_ and
_Unveil_ being the least fine-grained when compared to other sandboxing
mechanisms, this flexibility allows for creating a more secure sandbox than
other mechanisms.

Another case is _lowdown_ , which is a library that uses _Pledge_ and
_Capsicum_ to create a sandbox. Examining the source code reveals that the
developers had intentions to adopt _Seccomp_ but didn’t since there are checks
in the config files and tests for _Seccomp_ , but it’s not used in the actual
library code. _Pledge_ is used in two functions, sandbox_pre and sandbox_post
with stdio, rpath, wpath, and cpath promises in the sandbox_pre and only stdio
promise in sandbox_post. With _Capsicum_ , only the sandbox_post function is
implemented and _Capsicum_ is used to limit the process access to stdin,
stdout, and stderr with respective permissions and then entering the
capability mode with cap_enter. The developer was able to sandbox the program
in its early stages using _Pledge_ and specify system calls that are needed at
the start of the program and then drop privileges to only allow the stdio
promise but didn’t do that with _Capsicum_.

##  6\. Discussion

The results shed light on the adoption of sandboxing mechanisms and the way
developers use these mechanisms indicates that there is a space for
improvement and that the adoption of sandboxing mechanisms can be further
increased.

###  6.1. Sandboxed from the off

Examining the post-boot processes in the different operating systems shows
intriguing findings. As depicted in Table 5, OpenBSD stands out with 36
processes running after boot, of which 33 employ either _Pledge_ or _Unveil_.
Following that, Fedora has 16 processes out of 138 utilizing _Seccomp_ , while
Debian has 5 out of 120 processes that use _Seccomp_. With FreeBSD, only 3 out
of 47 processes utilize _Capsicum_. Although OpenBSD exhibits a significant
advantage over other operating systems in this context, this distinction is
less pronounced when considering software packages.

Whilst the number of packages that directly invoke sandboxing APIs seems to be
very low in all operating systems (OpenBSD has the highest percentage), the
number of packages that depend on them is much higher. Understanding whether
an app is appropriately sandboxed becomes difficult when it’s not immediately
apparent how a sandbox is deployed. For example, is it accurate to label
deepin-terminal as sandboxed simply because it relies on a chain of
dependencies, eventually reaching openssh-client, which is sandboxed using
_Seccomp_? While one might argue that this minimal dependency on openssh-
client brings a level of sandboxing for a specific aspect, the broader
application may still be vulnerable.

Despite these considerations, a more thorough understanding of these
sandboxing mechanisms can contribute to their improved adoption. Security
software comes with challenges that might affect its adoption (DeWitt, 2007),
and we believe there is a need to better understand the space of problems
presented by these mechanisms. But what are these problem(s)?

Table 5. Processes that make use of sandboxing features immediately on boot following a clean install. OS | Post-boot processes | Sandboxed
---|---|---
Debian | 120 | 5 (4.16%)
Fedora | 137 | 16 (11.67%)
OpenBSD | 36 | 33 (91.66%)
FreeBSD | 47 | 3 (6.38%)

###  6.2. Uncharted territory

One part of the problem appears to be the lack of understanding of the
developers’ mindset and rationale. What do developers look for in a sandboxing
mechanism? Are developers facing usability issues with the current ones? Do
developers use these mechanisms correctly?

Our results show that _Pledge_ and _Unveil_ , despite being developed
recently, have a higher usage percentage than _Seccomp_ in Debian or Fedora
packages. _Pledge_ and _Unveil_ have a very simple interface and are easy to
integrate with programs. There are efforts to port _Pledge_ and _Unveil_ to
Linux by open-source developers and this demonstrates an interest in using
these mechanisms.

When looking at _Seccomp_ and _Landlock_ , such mechanisms allow implementing
fine-grained sandboxes, but they need more work from the developers to apply
them when compared to _Pledge_ and _Unveil_. During our analysis, we noticed
that developers write a lot more code to adopt _Seccomp_ when compared to
other sandboxing mechanisms. For instance, gmid package has 549 lines of code
to implement _Seccomp_ sandboxing while only have 28 lines to adopt _Capsicum_
and 96 to adopt _Pledge_ and _Unveil_. We also noticed that a lot of packages
write hundreds of lines of codes as wrappers for _Seccomp_ API as in _brltty_
and _crun_ packages.

We noticed that sometimes developers copy _Seccomp_ filters code from other
packages as in _ansilove_ , _bdf2sf_ and _logswan_ that share similar code of
_Seccomp_ filters. In addition, some packages share a list of system calls
that are considered dangerous and should be denylisted, we noticed this in
_flatpak_ package where developers stated in the code comments that the
_Seccomp_ denylist has been copied from _linux-user-chroot_ which is
influenced by _sandstorm.io_ denylist. While in these cases, we didn’t notice
any issues with the shared code, this behavior is concerning as research shows
that developers tend to copy code without giving it any security
considerations (Fischer et al., 2017).

So is the problem that some sandboxing mechanisms are complex and not easy to
use? One would assume that when using _Seccomp_ , developers will tend to
avoid the complexities of writing BPF filters, but we noticed a lot of
packages use _Seccomp_ with BPF filters to scan system calls arguments. BPF
filters are difficult to write, yet developers are willing to go the extra
mile and implement a more secure sandbox even if it requires complex
implementations.

All of above begs the question of whether we really need more fine-grained
sandboxing or should operating systems maintainers focus on creating simple
sandboxing mechanisms. There is a general lack of understanding of what
developers need in a sandbox and the struggles they face when using the
current mechanisms. Understanding these areas will provide insights into how
the adoption of sandboxing mechanisms can be increased.

###  6.3. It is not easy

Still, even with simple mechanisms like _Pledge_ and _Unveil_ , the adoption
levels seem to be very low, there have to be other challenges developers are
facing. One problem is that in order for developers to use any of the studied
mechanisms, they have to have a solid understanding of what the application is
doing, what system calls it is using, and which files it needs access to. This
becomes more challenging when the program to be sandboxed is large and
complex. In addition, as seen in the cases we presented earlier, sometimes
developers have to re-architecture their applications and split them into
smaller units and processes to accommodate sandboxing. This burden can be one
of the reasons that developers steer away from adopting such technologies.

Furthermore, debugging _Seccomp_ and _Pledge_ is challenging. When using a
functionality outside the promised ones, the program will exit, and produce a
core dump, and the developer has to either analyze the core dump or debug the
program to figure out the violated promise. With _Seccomp_ , the developer has
to use external tools like _strace_ or write debugging logic to be able to
capture what went wrong.

###  6.4. But how can we fix this?

Research shows that developers prioritize functionality over security, and
they show indifference towards security (Arizon-Peretz et al., 2021; DeWitt
and Kuljis, 2006). One solution to this problem is to have programming
languages and platforms to implement security measures by default. One example
of this is Deno, a Javascript runtime environment that is used to build web
applications. By default, Deno doesn’t grant applications any access to system
resources, and the developer is required to explicitly ask for permissions to
the resources using an API (Deno, 2023). Such an approach doesn’t leave an
option for developers but to adopt security. Furthermore, having tools in
operating systems that allow developers to check if their application is over-
privileged can guide developers to utilize these mechanisms better.

##  7\. Related work

###  7.1. Adoption of security mechanisms

There is no research that focuses on the adoption of sandboxing solutions, but
some research investigated the adoption challenges of security tools in
general from both the social and the technical aspects. In their research,
Witschey et al. quantified developers’ adoption of security tools by
conducting an online survey with developers to determine the factors that
could determine security tools adoption (Witschey et al., 2015). They found
out that social and cultural factors such as interaction with the security
community and the reputation of the security tool represent the most important
factors that affect security tool adoption. In addition, their research
highlights many innovative factors that affect the adoption of security tools.
One of these factors is how policy-makers might be able to increase security
tool adoption by increasing observability. This means that developers are more
likely to use a security tool when they observe other developers using it.
This can be done with sandboxing by showing that certain packages use
sandboxing mechanisms to maximize the security of their application. Another
factor is the complexity, their survey found that complexity is usually linked
to the ability of the security tool and that security tool designers can
increase adoption by reducing the complexity of security tools.

Linden et al.’s research examines the rationale behind developers’ decisions
when it comes to decisions that affect the security of the application (Van
Der Linden et al., 2020). They conducted two studies and found that developers
might be writing secure code but not secure apps. Their research provided
insights that help in understanding developers’ rationale and they provided
suggestions to help developers make security-aware decisions, they suggested
enhancing development IDEs with features that incorporate support and help in
making the right choices when it comes to security features and mechanisms.

###  7.2. Research on sandboxing mechanisms

Previous research examined the sandboxing space and categorized sandboxing
solutions based on many factors. Maass et al. conducted a systematic
literature review of sandboxing solutions (Maass et al., 2016). Their research
discusses many definitions of the sandbox term and that it could be an
isolation or policy enforcement technique. The research discussed the lack of
validation techniques for present solutions and that most of the current
solutions focus on the performance overhead and they ignore validating other
important aspects such as the usability of the sandbox. The research provides
recommendations to better validate sandboxes such as using assurance case
diagrams. In addition, the paper discusses the lack of research on the
usability of sandboxing and provides insights into how the usability of
sandboxes can be assessed.

Schreuders et al. conducted a survey of sandboxing solutions focusing on
application-oriented solutions (Schreuders et al., 2013). They categorized
application-oriented solutions into rule-based and isolation based and went
further to discuss the limitations and challenges of the adoption of these
sandboxes. The research outlines that the challenges of the adoption of
studied sandboxes stem from their usability challenges and policy complexity.

###  7.3. Usability of security mechanisms

The research highlighted the importance of having usable security APIs and
that usability should be considered well when designing security software
(Yee, 2004; Green and Smith, 2016). Research indicates a strong correlation
between security and usability, emphasizing that disregarding usability can
have dire consequences (Acar et al., 2017). However, many security software
suffers from usability issues, and developers struggle to use security APIs
and software. Schreuders et al. examined the usability of three access-control
mechanisms, including SELinux, AppArmor, and FBAC-LSM by conducting a
usability study (Schreuders et al., 2012). Their results illustrated many
usability issues with SELinux and AppAromor and they proposed changes to these
mechanisms to avoid the found usability issues. DeWitt and Kuljis conducted a
usability study on Polaris (a security-related software that is designed to be
usable) (DeWitt and Kuljis, 2006). Their research found that developers showed
indifference towards security and that despite Polaris software being designed
to be usable, it still had many usability issues. Furthermore, their research
found that aligning usability with security becomes more challenging when the
usability aspect is considered later in the development process. Patnaik et
al. research examined the usability of cryptography APIs by looking at 2000
StackOverFlow questions which studied whether cryptography libraries violate
Green and Smith principles. They identified many usability smells in
cryptography APIs and grouped them into categories-_whiffs_ (Patnaik et al.,
2019).

##  8\. Conclusion

While sandboxing mechanisms play a pivotal role in enhancing security, our
findings reveal a nuanced landscape. Many packages incorporate sandboxing
mechanisms indirectly, yet only a limited number directly use these
mechanisms. The diversity of sandboxing mechanisms presents both opportunities
and challenges for developers aiming to implement secure software across
multiple platforms. The cases we presented suggest that, despite their
apparent simplicity, mechanisms like _Pledge_ and _Unveil_ are capable of
implementing strict sandboxes when compared to other fine-grained mechanisms.
Furthermore, our research highlights the lack of understanding of the
challenges that developers face when using these mechanisms and whether these
mechanisms are designed to fit developers’ needs in the first place.
Addressing these questions will provide further insights into how to drive
developers to use sandboxing mechanisms more and to better understand the lack
of adoption of these mechanisms in open-source ecosystems.

## References

  * (1)
  * Acar et al. (2017) Yasemin Acar, Michael Backes, Sascha Fahl, Simson Garfinkel, Doowon Kim, Michelle L Mazurek, and Christian Stransky. 2017.  Comparing the usability of cryptographic APIs. In _2017 IEEE Symposium on Security and Privacy (SP)_. IEEE, 154–171.
  * Arizon-Peretz et al. (2021) Renana Arizon-Peretz, Irit Hadar, Gil Luria, and Sofia Sherman. 2021\.  Understanding developers’ privacy and security mindsets via climate theory.  _Empirical Software Engineering_ 26 (2021), 1–43.
  * Beck (2019) Bob Beck. 2019.  Pledge and Unveil in OpenBSD.  _Open BSD project_ (2019).
  * Calavera and Fontana (2019) David Calavera and Lorenzo Fontana. 2019.  _Linux Observability with BPF: Advanced Programming for Performance Analysis and Networking_.  O’Reilly Media.
  * Canella et al. (2021) Claudio Canella, Andreas Kogler, Lukas Giner, Daniel Gruss, and Michael Schwarz. 2021.  Domain Page-Table Isolation.  _arXiv preprint arXiv:2111.10876_ (2021).
  * Deno (2023) Deno. 2023.  Deno permissions.  _Deno documentation_ (2023).
  * DeWitt and Kuljis (2006) Alexander J DeWitt and Jasna Kuljis. 2006.  Aligning usability and security: a usability study of Polaris. In _Proceedings of the second symposium on Usable privacy and security_. 1–7.
  * DeWitt (2007) Alexander John Anthony George DeWitt. 2007\.  _Usability issues with security of electronic mail_.  Ph. D. Dissertation. Brunel University, School of Information Systems, Computing and Mathematics.
  * Fischer et al. (2017) Felix Fischer, Konstantin Böttinger, Huang Xiao, Christian Stransky, Yasemin Acar, Michael Backes, and Sascha Fahl. 2017.  Stack overflow considered harmful? the impact of copy&paste on android application security. In _2017 IEEE Symposium on Security and Privacy (SP)_. IEEE, 121–136.
  * Green and Smith (2016) Matthew Green and Matthew Smith. 2016.  Developers are not the enemy!: The need for usable security APIs.  _IEEE Security & Privacy_ 14, 5 (2016), 40–46.
  * Kurchuk and Keromytis (2004) Aleksey Kurchuk and Angelos D Keromytis. 2004.  Recursive sandboxes: Extending systrace to empower applications. In _Security and Protection in Information Processing Systems: IFIP 18 th World Computer Congress TC11 19 th International Information Security Conference 22–27 August 2004 Toulouse, France_. Springer, 473–487.
  * Maass et al. (2016) Michael Maass, Adam Sales, Benjamin Chung, and Joshua Sunshine. 2016.  A systematic analysis of the science of sandboxing.  _PeerJ Computer Science_ 2 (2016), e43.
  * OpenBSD documentation (2021) OpenBSD documentation. 2021\.  Unveil system call documentation in OpenBSD.  _OpenBSD project_ (2021).
  * OpenBSD documentation (2022) OpenBSD documentation. 2022\.  Pledge system call documentation in OpenBSD.  _OpenBSD project_ (2022).
  * Patnaik et al. (2019) Nikhil Patnaik, Joseph Hallett, and Awais Rashid. 2019\.  Usability Smells: An Analysis of $\\{$Developers’$\\}$ Struggle With Crypto Libraries. In _Fifteenth Symposium on Usable Privacy and Security (SOUPS 2019)_. 245–257.
  * Polo (2021) Omar Polo. 2021\.  Comparing sandboxing techniques.  https://www.omarpolo.com/post/gmid-sandbox.html
  * Salaün (2017) Mickaël Salaün. 2017\.  Landlock LSM: toward unprivileged sandboxing.  _Linux Security Summit_ (2017).
  * Saltzer and Schroeder (1975) J.H. Saltzer and M.D. Schroeder. 1975.  The protection of information in computer systems.  _Proc. IEEE_ 63, 9 (Sep 1975), 1278–1308.  https://doi.org/10.1109/PROC.1975.9939
  * Schreuders et al. (2012) Z Cliffe Schreuders, Tanya McGill, and Christian Payne. 2012\.  Towards usable application-oriented access controls: qualitative results from a usability study of SELinux, AppArmor and FBAC-LSM.  _International Journal of Information Security and Privacy (IJISP)_ 6, 1 (2012), 57–76.
  * Schreuders et al. (2013) Z Cliffe Schreuders, Tanya McGill, and Christian Payne. 2013\.  The state of the art of application restrictions and sandboxes: A survey of application-oriented access controls and their shortfalls.  _Computers & Security_ 32 (2013), 219–241.
  * Van Der Linden et al. (2020) Dirk Van Der Linden, Pauline Anthonysamy, Bashar Nuseibeh, Thein Than Tun, Marian Petre, Mark Levine, John Towse, and Awais Rashid. 2020.  Schrödinger’s security: opening the box on app developers’ security rationale. In _Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering_. 149–160.
  * Watson et al. (2010) Robert N M Watson, Jonathan Anderson, Kris Kennaway, and Ben Laurie. 2010.  Capsicum: practical capabilities for UNIX. In _19th USENIX Security Symposium_. 17\.
  * Witschey et al. (2015) Jim Witschey, Olga Zielinska, Allaire Welk, Emerson Murphy-Hill, Chris Mayhorn, and Thomas Zimmermann. 2015.  Quantifying developers’ adoption of security tools. In _Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering_. 260–271.
  * Yee (2004) Ka-Ping Yee. 2004\.  Aligning security and usability.  _IEEE Security & Privacy_ 2, 5 (2004), 48–55.
  * Zaborski (2016) Mariusz Zaborski. 2016\.  Capsicum and casper-a fairy tale about solving security problems.

  *[LSM]: Linux Security Module