# Extricating IoT Devices from Vendor Infrastructure with Karl

Gina Yuan, David Mazières, Matei Zaharia
Stanford University

###### Abstract

Most consumer IoT devices are vertically integrated with cloud-side
infrastructure. Such architectures present enormous risk to user data,
exacerbated by vendor heterogeneity and the inability for users to audit
cloud-side activity. A more promising approach would be to leverage local
hardware, providing users control over how their data is processed and why it
can be shared with other devices or the Internet.

Karl is a new smart-home framework designed to host IoT computation and
storage on user-chosen devices. A key insight in Karl’s modular programming
model is that a familiar interface (inspired by serverless) can capture most
modern cloud-side IoT components under a single framework, which executes
modules agnostic of hardware location. While local hosting eliminates many
flows, modularity enables all remaining flows to be justified using fine-
grained primitives. We introduce two IoT security mechanisms: _pipeline
permissions_ that permit device data to be shared given some justification and
_exit policies_ that block flows unless specific conditions are met. We
evaluate Karl through two end-to-end applications.

##  1 Introduction

This paper presents Karl, a new smart home framework that eliminates
dependence on vendor services by making local hosting practical. The tight
integration between vendor services, companion apps, and modern IoT devices
has increased the size of the smart home attack surface [2]. For example, most
smart speakers parse audio commands such as “turn on the light” using a cloud
service, even when they pertain only to local devices; doorbell cameras send
video to the cloud; and even smart locks work through cloud services. Many
devices are hard to update or lose vendor support over time, leaving them with
software vulnerabilities. The need for devices to interoperate through “hubs”
like HomeKit [16, 49, 4] can enable cross-device attacks. Overall, the
dependency on vendor services has led to a large range of attacks on smart
home devices [33, 43, 8, 47, 23, 34, 22, 3, 39].

(a) The modern IoT ecosystem relies heavily on remote hardware.

(b) Karl executes most functionality on local hardware.

Figure 1: Division of device functionality on local and remote hardware. In
this example, the smart speaker handles two speech commands: one that asks the
weather and one that turns on a light.

Theoretically, an architecture in which user data never leaves user hardware
offers better privacy and security. For many functions, such as turning on the
light by voice, there is no need to send data remotely. Indeed, enterprise-
grade IoT devices often have purely local control. Unfortunately, the consumer
setting creates several challenges to local hosting: Devices are typically
inexpensive and computationally constrained. Home networks prevent incoming
connections, making it hard to connect companion apps directly to a device
such as a camera. Users are less sophisticated, unable to configure servers or
firewalls, and have difficulty understanding the security implications of
configuration choices. And of course the lower price point of consumer devices
makes almost any level of individual customer support untenable. The result is
an over-reliance on vendor services, to the point that even locally hosted
controllers such as Home Assistant [26] have a depressingly high fraction of
integrations going through the cloud to communicate with local devices.

(a) Example dataflow graph of a smart speaker.

Pipeline Permission
speaker.speech_command $\rightarrow$ speech_to_intent $\rightarrow$ weather
$\rightarrow$ weather.com

Exit Policy
Tag: speaker.speech_command
Exit Policy: speech_to_intent

(b) A generated pipeline permission and a user-specified exit policy.

Figure 2:  The pipeline permission shows that the speaker wishes to send all outgoing data through the speech_to_intent module, which converts speech to text intents; then weather-related intents go to weather.com. Separately, the user could add an exit policy saying “all speech commands need to go through speech_to_intent”. Category | Justification | Smart Speaker Example
---|---|---
Offload computation | weak local hardware | ML e.g., speech to intent
Offload storage | scalability, fault tolerance | record notable audio events
Remote access | minimize system administration of NAT and firewalls | listen to remote audio feed
Pull data | data not available at production, Internet-scale knowledge | weather queries, firmware updates
Push data | aggregate analytics on data | bug reports, training data, analytics
Table 1: Classifying device network accesses and their justifications.

Fig. 1 shows Karl’s architecture. Karl leverages local hardware to move
processing and storage out of devices and centralized vendor services and onto
a user-controlled device such as an old PC. It offers a simple but expressive
programming model based on modules that process data, such as speech-to-
intent, and communicate with each other through a key-value data store. Each
module executes in isolation, similarly to a serverless function [6], and each
data and network access is validated by the locally-hosted Karl hub. The hub
also serves Karl apps directly to the user’s phone.

While existing IoT hubs enable access control between different devices and
the Internet, this has proven too coarse-grained [12, 20, 35, 16]. For
example, SmartApps are always granted full access to a device even when they
only require limited access [20]. Karl, by contrast, enforces security at the
module granularity. It represents the smart home as a dataflow graph, where
device data flows through modules to other devices and the network (Fig.
2(a)). Users interact with the graph through a UI that visually represents the
policy. Most importantly, whereas users cannot audit cloud services or enforce
their terms of service [30, 17], Karl can enforce the policies it presents to
users on all locally hosted code.

While local hosting eliminates many data flows, modularity enables all
remaining flows to be justified using fine-grained primitives. Karl provides
two security mechanisms (Fig. 2(b)): First, Karl automatically generates
easily-reviewable _pipeline permissions_ that describe why device data can
flow to another device or to the network (e.g., the speaker must share a
location to query the weather). Second, Karl allows users to set _exit
policies_ on data to block flows unless specific conditions based on modules
are met (e.g., data derived from raw audio can only exit if it has first been
transformed to a textual intent). Compared to existing IoT frameworks, Karl
can provide more ubiquitous and meaningful privacy guarantees.

Karl tackles four major challenges to achieve its goals of eliminating
dependence on vendor services and enabling expressive privacy policies on all
remaining network accesses:

#### Supporting Complex Device and Service Interactions

The first challenge is accounting for the entire attack surface of modern
devices (Table 1). Consider a smart speaker such as the Amazon Alexa or Google
Home. Due to hardware constraints, the speaker must offload some computation
and storage. When the user interacts with the speaker through a companion app,
they must go through a cloud endpoint mediated by the vendor. The speaker may
also download firmware updates, share analytics, or otherwise pull and push
data to the Internet. Each network interaction adds a security risk.

Existing frameworks were not designed for the network security demands of
modern IoT. Industry hubs like SmartThings [49] and research frameworks built
on top of it [52, 31] still require vendors to host their own services. For
example, Alexa’s SmartThings integration serves only to connect to Amazon
servers. Frameworks like FlowFence [21] mediate remote connections through the
hub, but still permit flows to leak data when not fundamentally required. For
example, FlowFence explicitly allows sharing a door lock’s state with the
Internet since it is required for the mobile companion app.

In comparison, Karl moves modern functionality to trusted hardware under a
simple but expressive programming model. Karl executes its modules locally and
also hosts the data store on local hardware. The Karl hub also runs and serves
companion apps directly to a user’s phone through a user’s router. Flows that
use an external service, such as downloading firmware updates, can still
contact those services but only if the user allows them with Karl policies.
These basic abstractions enable most device-to-device and companion app
interactions to run on Karl-controlled hardware.

#### Usable Privacy Policies

The second challenge is expressing privacy policies when a device must access
the Internet. Existing techniques are either obfuscated in cloud
infrastructure or based on coarse-grained security primitives such as device-
to-device access control. In comparison, local hosting enables Karl to enforce
ubiquitous policies that consider all components of modern IoT functionality,
as opposed to blindly trusting multiple vendors. Modules also enable finer-
grained privacy policies with semantic meaning such as requiring audio data to
be converted to text. Thus Karl’s programming model empowers users to express
and enforce policies that correspond to more intuitive privacy guarantees.

We additionally co-design functionality and policies in Karl such that
modifying one reflects in the other. In particular, Karl enforces user privacy
policies by modifying the dataflow graph that represents device functionality
to comply with pipeline permissions and exit policies, such as by disabling
network access or deleting an edge. Karl treats the generated graph as a
declarative policy, automatically labeling data with non-hierarchical tags in
the data store and enforcing mandatory access control on the edges. This
mechanism combines the power of information flow with the simplicity of access
control to enforce useful privacy policies at a fine granularity.

#### Performance

A third challenge is maintaining performance for real-time interactions and
compute-heavy tasks without amortizing costs on cloud hardware. By using local
hardware, Karl can minimize latencies for many interactions [45] and even
outperform cloud-based services. For heavy compute tasks such as ML, which
typically run as long-lived services in the cloud, Karl accounts for the high
initialization times of its stateless modules using a variant of speculative
execution. Overall, we find that Karl has acceptable performance.

#### System Administration

Though Karl is best hosted on local hardware for privacy, end users may prefer
the durability and scalability of the cloud. We show it is feasible to deploy
Karl in the cloud but on user-managed hardware, a similar approach to DIY
Hosting [40]. The cloud deployment increases network latency and costs money,
but self-hosting Karl in the cloud (e.g., using Amazon Web Services) is still
comparable in cost to popular IoT cloud subscriptions, and improves privacy
because the user need only trust their own cloud provider rather than vendors
and their providers.

We implement and evaluate a functioning prototype of Karl. We port 4 home IoT
devices—a security camera, smart speaker, light bulb, and occupancy sensor. We
implement several applications in which these devices interact with each other
and third parties outside the home network. We analyze several privacy
properties that users may desire but cannot be enforced in existing
frameworks, and demonstrate how they are intuitive to express in Karl. In
addition, we demonstrate that Karl offers reasonable performance and can
improve the latency of some interactive applications. Finally, we analyze the
tradeoffs of self-hosting Karl in the cloud.

In summary, our contributions are:

  * •

We introduce Karl’s modular programming model, which makes it practical to
support the rich interactions of modern devices on local, user-managed
hardware.

  * •

We introduce pipeline permissions, which restrict devices to justifiable data
flows based on finer-grained primitives with semantic meaning.

  * •

We introduce _exit policies_ as a way to capture end-to-end confidentiality
policies users care about and avoid unintended consequences of pipeline
permissions.

  * •

We present and evaluate an implementation of Karl on applications spanning 4
devices and 10 modules.

##  2 Security Properties

The stakeholders in the Karl ecosystem are hardware vendors, module
developers, and end users. We assume end users properly install Karl,
configure secure login credentials (through the Karl hub UI), and that Karl
can mediate all Internet traffic. Some hardware vendors and module developers
may be compromised or negligent, however, leading to vulnerable components.
Attackers attempt to leverage these vulnerable components—possibly in
collusion with their compromised creators—so as to violate end-user privacy or
actuate smart-home controls in undesired ways.

Karl’s security guarantee is that it restricts inter-component communication
to the dataflow graph specified by the user’s pipeline permissions and exit
policies. The benefit of this model is that the presence of a few secure
modules can significantly restrict the consequences of compromised components.
For instance, the only path through which training data can leave the network
may go through a “statistics” module that restricts communication to a trusted
vendor’s analytics endpoint. In this case, a camera with malicious firmware
cannot leak video to outside parties on the wide-area network.

Some vendors may host cloud services, but this will be apparent in the
pipeline permission. In many cases, the communication can be mediated by
simple, well-known, and trustworthy modules that provide meaningful security
guarantees. For example, firmware updates should be mediated by a data pump
module that ensures devices receiving firmware cannot send large amounts of
locally recorded data back upstream.

Pipeline policies enable visual verification that devices gain approximately
least privilege, simplifying the process of determining that the permissions
suit the functionality. In practice, only a small fraction of users may
actually object to unwarranted permissions, but the hope is that this is
enough to flag dangerous devices for the rest of the ecosystem.

Karl does not prevent covert channels that leak information through resource
utilization. Nor does it prevent side-channel attacks, such as cache timing,
microarchitectural data leaks, or network timing attacks. However, such
attacks are typically difficult to disguise in plausibly legitimate source
code. Hence, we assume most hardware vendors and module developers are well-
intentioned with reputations to uphold, and would not risk wall-banging
attacks in production code.

##  3 Motivating Examples

To motivate the need for a modern smart home framework with strong privacy
guarantees, we start with a comparison of popular IoT frameworks in industry
(Samsung SmartThings), the open-source community (Home Assistant), and
research (FlowFence). We base our comparison off the smart speaker example
from the introduction and discuss several privacy properties a user might
desire. For each framework, we analyze how a developer would implement the
smart speaker, and how the user would enforce the following privacy
properties:

  1. 1.

The speaker can share data with weather.com if I ask about the weather, but
not the raw audio.

  2. 2.

The speaker can share data with the light if I ask it to turn the light on or
off, but not the raw audio.

  3. 3.

No one (except me) should access raw audio data unless it has been transformed
from speech to an intent.

  4. 4.

The vendor can’t switch my light on or off.

In existing frameworks, developers bear the burden of protecting user data,
and users cannot verify their privacy guarantees:

#### 1) Samsung SmartThings.

SmartThings [49] is an industry home automation framework that can connect
hundreds of brands and thousands of devices. The platform is centered around
the SmartThings Cloud, which remotely manages devices, data, automations, and
more. Devices connect to the Cloud through a phone, hub, WiFi, or third-party
cloud. Vendors write integrations for simple devices such as a light bulb or
sensor based on a schema that also provides authentication and a consistent
UI. Devices with more complex automation logic, such as a smart speaker, must
implement SmartApps, connectors that run on AWS Lambda or another vendor-
hosted server and send events back to the Cloud.

#### Vendor Perspective.

Amazon Alexa and Google Home are both smart speakers with SmartThings
compatibility. However, they primarily use SmartThings to proxy commands to
hub-connected devices that use different protocols or schemas. The devices are
still connected to their original cloud services, which they depend on for
remote access and tasks like speech-to-text. While larger companies may
appreciate the flexibility of being able to integrate their existing cloud
infrastructure, smaller vendors face high liability and startup costs.

#### User Perspective.

Allowing devices unrestricted network access requires total trust in the
vendor. Network traffic is typically encrypted, so privacy settings in the
companion app are not verifiable. Raw audio data could be leaked through
vulnerabilities in the smart speaker, SmartApp, Cloud, or the light bulb the
speaker turns on. The user can restrict communication between different
devices through the SmartThings Cloud, but devices that require additional
cloud communication to provide functionality are outside the scope of the
framework. None of the four properties can be enforced.

#### 2) Home Assistant.

Home Assistant (HA) [26] is an open-source smart home framework with a focus
on local control and privacy. The HA architecture depends on an event bus that
fires and listens to events such as state changes and services. Developers
write integrations (e.g., image processing, light, Z-Wave) that extend the
core architecture with small pieces of home automation logic. Integrations are
limited to Python scripts and can access the network. HA is highly
programmable, and users have the option of self-hosting the framework on local
hardware such as a Raspberry Pi.

#### Vendor Perspective.

HA allows vendors flexibility to execute functionality locally or in the
cloud. HA offers smart speaker functionality through a combination of multiple
integrations. Almond [11] is an open, privacy-preserving virtual assistant
integration that acts as the text-to-intent backend. Almond also has a
repository of apps, called Thingpedia, such as an app that retrieves the
weather. Other integrations handle speech: Ada [46] is powered by Microsoft
Cognitive Services, and Rhasspy [42] is an offline voice assistant. Thus
smaller vendors can easily contribute to some or all parts of a device simply
by writing software integrations. HA is also compatible with Alexa and Google
Home, which similarly use HA as a hub to access devices like in SmartThings.

#### User Perspective.

Programmability and the option to self-host make it feasible for HA to
implement the majority of functionality on local hardware, but in the
remaining cases it is hard to verify privacy guarantees. HA does not sandbox
integrations. For example, Ada necessarily talks to the Microsoft API, but it
is unclear if or when other integrations such as a speech-to-intent
integration access the network. The combination of services, state changes,
and other events make it difficult to manually trace which data might have
been exfiltrated and where. In addition, part of the dataflow for asking the
weather is in a non-ephemeral Almond server rather than an HA integration.
Devices such as the Alexa have similar privacy concerns as in SmartThings.
Thus while the user can take steps to run mostly privacy-preserving software,
it is still difficult to justify every network access that may contain data.
None of the four properties can be enforced.

#### 3) FlowFence.

FlowFence is a research framework for smart homes that takes a dataflow
approach to privacy instead of access control. In FlowFence, IoT apps consist
of functions that compute on sensitive data (quarantined modules or QMs), and
code that does not compute on sensitive data. QMs execute inside Java
sandboxes on the hub, which can run on an Android phone. QMs access sensitive
data via opaque handles, monitored by the hub according to flow policies.
Also, QMs communicate via event channels or a key-value store. Developers
declare intended dataflows of sensitive data to other devices or the network,
which users approve via UI prompts when a flow is first required.

#### Vendor Perspective.

It is straightforward for vendors to modularize existing device-side logic and
port it to QMs in FlowFence, similar to other app-based frameworks. However,
FlowFence does not explictly consider how to integrate companion apps into the
framework, such as to remotely turn on a light, as evidenced by the example
flow that sends door state to the Internet for the user to view. It also does
not consider how to express policies that use cloud services such as machine
learning, which are not as easily modularized while preserving performance
given high initialization times, and may not fit within the Java programming
model. Vendors must also consider how to set and interpret taint labels within
module code. Thus FlowFence complicates device-side logic while still
requiring vendors to manage their own infrastructure.

#### User Perspective.

Unlike the previous frameworks, FlowFence requires all network connections
involving sensitive data to go through the hub. The user can enforce P1 and P2
by approving flows that correspond to the properties. However, it may be
difficult for users to understand and approve every flow, particularly because
a device also includes flows for cloud services and companion apps, which can
negate P4. FlowFence does not have the concept of restricting categories of
flows such as in P3. Furthermore, some of the suggested flow policies are
overly permissive because they do not include the application semantics in the
QM pipeline. For example, the flow from a speaker to the lock means a
malicious QM could send raw audio to the lock, even if only a subset of that
information is required.

#### 4) Karl.

Karl provides privacy through local hosting that captures all the rich
interactions that modern IoT devices have with cloud services, companion apps,
and other devices; and modularity that enables enforceable, fine-grained
privacy policies. Karl’s programming model is based off the familiar
serverless interface that is easy for existing vendors to adopt.

#### _Vendor Perspective._

The vendor splits device functionality into firmware for the device, and
modules that cannot run on the device, because they require too much
computation or storage or because they run on a user’s phone. The vendor
programs an initial dataflow graph into the firmware, representing the
device’s functionality. Instead of creating a mobile app, the vendor creates a
Karl app, which the hub serves from the user’s machine to the user’s phone.
The hub downloads modules from a Karl package manager and executes the modules
on local hardware. The vendor no longer needs to host any cloud infrastructure
or user data.

#### _User Perspective._

The user installs a device, which locates the Karl hub on the local network,
and sends the initial dataflow graph. The user gets a notification that a new
device is available and approves it. As part of approving the device, the user
reviews a set of pipeline permissions corresponding to P1 and P2, showing
which device data may be shared and why. The user may also assign exit
policies such as P3 to device and application data, preventing sensitive data
from being shared under certain conditions regardless of pipeline permissions.
The user is guaranteed P4 due to local hosting of companion apps, since the
light does not need network access.

In the following sections, we describe the programming model for Karl and
illustrate how Karl can enforce novel privacy guarantees that existing
frameworks cannot.

##  4 Programming Model

Karl’s programming model is intentionally similar to existing event-driven
programming models to provide a familiar interface to IoT developers. Unlike
existing programming models, Karl’s is designed to eliminate vendor dependence
on cloud infrastructure—cloud services, companion apps, hardware—using the
simplest abstractions for compute and storage.

In this section, we describe our abstractions for compute and storage, and the
Karl app UI. Karl expresses all functionality as modules that interact with a
persistent data store and Karl web apps. We then describe the dataflow graphs
that represent device functionality, and discuss example graphs for three
different IoT devices that handle sensitive data.

#### Serverless modules.

Field |  Value
---|---
Name |  light_switch
Inputs |  intent – JSON of the form { type: “light", state: <state> }, where the state is “on" or “off"
Outputs |  state – $1$-bit to turn on and $0$-bit to turn off
Domains |  N/A
Table 2: Example specification of the light_switch module that maps a JSON
intent to a state the light bulb can understand.

Similar to the serverless programming model, a module is a self-contained
program that executes code given some inputs. Modules can take multiple inputs
and return multiple outputs. Each input and output is named and associated
with a particular data type, and is part of the module specification (Table
2). The module must also specify any domain names with which it requires
network access.

Modules execute inside sandboxes managed by the Karl hub. The module has a
single network connection to a sandbox, which proxies data and network
accesses (Listing LABEL:lst:module-sdk) to the data store and the Internet.
The sandbox decides which accesses are allowed, forwarding data accesses to a
controller and HTTPS requests to the requested domain. Note that data accesses
are keyed by the names in the module specification, agnostic of how data is
used outside the module.

⬇

class ModuleAPI:

def read(input, lower_timestamp, upper_timestamp)

def read_last_n(input, n)

def read_event(input)

def push(output, bytes)

def network(domain, request)

⬇

\end{lstfloat}

⬇

import karl

intent = karl.read_event("intent")

if intent["state"] == "on":

karl.push("state", [1])

else:

karl.push("state", [0])

⬇

\end{lstfloat}

(a) Smart light.

(b) Smart speaker.

(c) Security camera.

Figure 3: Example dataflow graphs. The gray boxes are devices and white boxes
are modules. The smaller boxes inside are input and output nodes. Edges
connect nodes and represent data dependencies. Solid edges are stateless,
while dashed edges are stateful. Blue edges flow to a module, while red edges
flow to a device. Green headers represent modules with network access. The
clock indicates a fixed interval schedule.

#### Persistent data store.

The _data store_ extends the key-value store, a common data structure for
stateful FaaS [48, 50]. In our data store, keys are tags that represent non-
hierarchical categories of data in the global namespace, and values are
append-only logs indexed by timestamps. While the module API operates in terms
of named inputs and outputs, the data store understands values in terms of
tags. The controller automatically assigns tags to data when modules read
inputs and push outputs (Listing LABEL:fig:module-spec-implementation), based
on the dataflow graph (§4.1).

The purpose of the data store is to provide a simple but familiar abstraction
that allows devices to replicate application logic. Stateless event-driven
programming is the most common communication pattern in existing frameworks:
SmartThings subscribe API, Home Assistant event bus, FlowFence event channels.
To provide stateful communication, existing frameworks may use state or key-
value stores associated with the device, or vendor-hosted cloud storage.

We are able to express a variety of stateful applications using the data store
as the only storage abstraction. Karl enforces access control rules on tags
based on the user’s privacy policies to prevent data leakage between different
modules. To leverage the privacy properties of stateless applications, which
are ephemeral and cannot leak data between invocations, the data store
additionally limits these modules to reading just the event for a tag that
caused the module to be spawned.

#### Karl web apps.

The Karl hub includes a per-user web interface for reading and pushing to the
data store, and configuring and spawning modules. Vendors can write static
pages with JavaScript (i.e. single-page apps) called Karl apps that wrap these
features to replicate the functionality of mobile companion apps. For example,
instead of serving sensitive photos from vendor-managed cloud storage, the
page can visualize data associated with the corresponding tag in the data
store. Instead of forwarding a request to view a live video feed through the
cloud, the page can spawn a module that turns on the livestream feature and
then visualize data associated with a livestream tag. It would be a simple
extension to also support a stateful webserver running as a module.

Karl apps allow vendors to provide the equivalent of companion apps directly
through the framework, without hosting cloud infrastructure. Typically,
vendors must host cloud services with longstanding connections to each device
to evade NAT. Alternatively, vendors roll their own web interfaces on the
device. Combined with port forwarding and insecure authentication, IoT devices
can easily expose the home network to security risks. Thus Karl apps improve
privacy and security compared to the existing state of companion apps.

###  4.1 Dataflow Graph

Each IoT device proposes an initial dataflow graph where data flows from the
device, through modules, and to other devices and the Internet (Fig. 3).
Though we discuss a visual graph, devices send their graphs to the hub using
JSON. Karl downloads the requested modules from a Karl package manager.
Vendors can either upload their own modules or use existing modules based on
their specifications.

The boxes in the dataflow graph are modules and devices, while nodes are their
inputs and outputs. For example, the light bulb in Fig. 3(a) outputs its state
and intensity, and has inputs that change the same state. Each node
corresponds to a tag, and edges represent stateless or stateful data
dependencies between nodes. When the set_true module pushes to its output that
is connected to a single input, the controller automatically adds the tag for
each node: set_true.true and #light.state. Modules spawn on three different
schedules: when data is pushed to a stateless edge, at a fixed time interval,
or when manually spawned by the user. Some modules require network access,
which implies a flow to the Internet.

In addition to data processing flows, the graph includes the flows needed for
Karl apps. Fig. 4(a) is the light bulb’s Karl app. The app visualizes the
light state and intensity by reading the sensor output tags. It sets the
intensity by pushing to the #light.intensity input. It switches the light on
or off by spawning a module that pushes a $1$-bit or $0$-bit to the #light.on
input. These interactions go through user hardware, so the light bulb state is
guaranteed to be private.

(a) Smart light.

(b) Security camera.

Figure 4: Example Karl apps that correspond with the dataflow graphs in Fig.
3. The user can set and visualize light state. In the camera, they can see
motion detection events and toggle the livestream.

We provide example dataflow graphs for two, more complex devices: a smart
speaker (Fig. 3(b)) and a security camera (Fig. 3(c)). The smart speaker
requires network access to pull weather data, while the camera pulls firmware
updates and pushes analytics. The smart speaker also expresses control flow
logic using modules. The camera uses a stateful edge to process a query over
recorded motion events, and spawns a firmware update module using a fixed
interval schedule.

The initial device dataflow graph provides a starting point for users who
introduce new devices to their home. Users can configure devices to interact
with each other, such as by enabling the smart speaker to turn on the light
bulb with a voice command. The user can also add privacy-preserving modules
between sensitive data and the network to reduce the fidelity of data before
sharing it with the Internet. We imagine a frontend for privacy settings and
automation logic, such as IFTTT [28] or the Home Assistant frontend, that
automatically maps user configurations to the underlying dataflow graph. For
example, the frontend could suggest edges based on corresponding data types or
recognize devices from the same vendor based on overlapping graphs.

##  5 Privacy Policies

The modular dataflow graph provides a convenient foundation for privacy
policies based on fine-grained application semantics. Karl can regulate the
flow of sensitive data from source to sink based on the semantics of the
pipeline of modules in between. Recall the justifications for IoT network
access in Table 1. Karl can provide compute, storage, and connectivity using
local hardware, but devices should still justify when they must fundamentally
pull or push data to the Internet.

Karl provides the framework for users to monitor these dataflows based on two
concepts. Pipeline permissions specify why and to whom data can flow based on
the pipeline of modules from source to sink. They serve to ensure that
dataflows match users’ intuitions, and that devices and modules do not engage
in unnecessary communication. However, seemingly plausible pipeline
permissions can still have unintended consequences. Exit policies capture end-
to-end data policies that block flows unless specific conditions based on
modules are met, regardless of pipeline permissions.

In this section, we discuss how the privacy properties in Fig. 5 map to
pipeline permissions and exit policies, apply these concepts to modify the
dataflow graph as a declarative policy, and describe how Karl compiles the
graph to a non-hierarchical form of mandatory access control.

  1. A.

The smart speaker can talk to the weather station if I ask about the weather,
without sending the actual recording.

  2. B.

The camera can share training data with the vendor.

  3. C.

The smart speaker can turn on the light if I tell it to, and without sending
the raw audio.

  4. D.

The camera can update its firmware from the trusted distribution URL.

  5. E.

The camera can share training data with the vendor only if I am not at home.

  6. F.

The camera can share training data with the vendor only if it is anonymized
such as by using Prio [13].

  7. G.

The light should never need to send data to the Internet.

  8. H.

Only I should ever be able to view the livestream.

  9. I.

The smart speaker can only send data derived from raw audio if it has first
been translated to text.

  10. J.

The camera can only send data derived from raw photos if it has been
anonymized.

Figure 5: Example privacy properties a user might desire.

###  5.1 Pipeline Permissions

  1. A.

speaker.speech_command $\rightarrow$ speech_to_intent $\rightarrow$ weather
$\rightarrow$ weather.com

  2. B.

camera.motion $\rightarrow$ person_detection $\rightarrow$ statistics
$\rightarrow$ statistics.com

  3. C.

speaker.speech_command $\rightarrow$ speech_to_intent $\rightarrow$
light_switch $\rightarrow$ #light.state

  4. D.

firmware.com $\rightarrow$ firmware $\rightarrow$ #camera.firmware

Legend=device.output, module, domain_name, #device.input

Figure 6: Pipeline permissions that correspond to privacy properties A-D in
Fig. 5, detected from the dataflow graphs in Fig. 3.

  1. E.

camera.motion $\rightarrow$ person_detection ($+$ occupancy_sensor.at_home)
$\rightarrow$ boolean $\rightarrow$ statistics $\rightarrow$ statistics.com

  1. F.

camera.motion $\rightarrow$ person_detection $\rightarrow$ prio $\rightarrow$
a.statistics.com
$\cdots$ $\rightarrow$ prio $\rightarrow$ b.statistics.com
$\cdots$ $\rightarrow$ prio $\rightarrow$ c.statistics.com

Legend=device.output, module, domain_name, #device.input

Figure 7: Pipeline permissions that correspond to privacy properties E and F
in Fig. 5. E and F are variants of B if the security camera had included
client-side anonymization in its graph.

When a user registers a device with Karl for the first time, appending its
dataflow graph, they must approve any new sensitive dataflows as pipeline
permissions (Fig. 6). These permissions ensure there are not any unintended
effects from introducing a new device into a complex ecosystem. Karl inspects
the graph for any new linear paths that exfiltrate device data to the network
(or other devices), or that send data to the device from the network. Karl
presents the textual form of these subgraphs to users as permissions to
review. These permissions represent tradeoffs between privacy and
functionality, and correspond to intuitive privacy properties.

The user considers the list of pipeline permissions generated by Karl based on
module semantics. Consider a home with a smart speaker, security camera, and
light bulb. Each device dataflow graph is appended as a disjoint graph. An
automation frontend links light_switch.state to #light.state, enabling the
user to turn on the light using a speech command. The permissions in Fig. 6
correspond to privacy properties A-D in Fig. 5. Permission A explains that the
user must send at least text to the weather station to learn the weather,
while permission D allows a third party to update the camera firmware because
it is a trusted vendor.

The user may reject a pipeline permission if after inspection, it does not
meet their privacy expectations. For example, given permission B, the user may
not think it necessary to share statistics that could include raw image data.
To gain the user’s trust, the vendor might add a module in the pipeline to
make it clear that they use data in an anonymized form such as E or F (Fig.
7). There are many such techniques for client-side protection of data [29, 13,
18, 10, 32]. In this case, the user may believe it is worth it to share
anonymized data because it improves the product for future use.

Note that the dataflows that access the network need only be those that
fundamentally pull and push data. This reduces the number of policies the user
must consider. For example, rather than sending the light state to the
Internet to visualize in the companion app, the state goes directly to the
Karl app and the flow does not need to be authorized. Rather than sending raw
audio to a cloud service to process the speech command, Karl translates the
audio locally.

It may be difficult for users to accurately judge every permission, a common
issue in user-driven policies [19, 44]. It will be important to conduct user
studies to determine how best to present pipeline permissions in the UI. But
even if the user approves all dataflows, Karl can still provide auditable logs
because it captures every network access and its provenance. To provide
additional reassurances about sensitive data, we combine the allow approach of
pipeline permissions with the deny approach using exit policies, in the next
section.

###  5.2 Exit Policies

| Tag | Exit Policy
---|---|---
G. | light.state | false
H. | camera.livestream | false
I. | speaker.speech_command | speech_to_intent
J. | person_detection.image | $\texttt{boolean}\ |\ \texttt{prio}$
Table 3: Exit policies proposed by a user for the devices in Fig. 3,
corresponding to privacy properties G-J in Fig. 5. The policy defaults to true
if unspecified, deferring to pipeline permissions.

Even if a user mistakenly approves a pipeline permission, modularity enables
the user to define exit policies on categories of data (Table 3). Karl’s data
store groups data under tags corresponding to the device it comes from or its
downstream modules. The user then defines conditions under which this data
category can be exfiltrated to the network or another device. These conditions
are at the granularity of Karl’s serverless modules, enabling expressive,
high-level policies.

Tags enable data-centric policies on important categories of data, rather than
policies on devices and apps that use the data. In Karl, devices distinguish
the different types of data they push at the source, such as camera.motion or
camera.livestream. Tags also represent the inputs and outputs of downstream
modules, such as indoor_person_detection.training_data for outputs of a module
that runs person detection specifically on indoor cameras. This distinction
between tags is built into Karl’s modular programming model, tightly
integrating vendor-defined functionality with policy enforcement.

We provide a simple policy language defined in terms of modules for specifying
conditions under which tags can be exfiltrated. The language consists of three
operators: $\&$, $|$, and $>$. The $\&$ and $|$ operators represent
conjunctions and disjunctions, while $>$ represents ordering. Note that true
implies the exit policy is met in all conditions and allows all pipelines,
while false implies the opposite. We expect this policy language to be
expressive and intuitive to developers, though a simpler alternative could use
only singular modules.

For example, defining the exit policy $(\texttt{boolean}\ |\
\texttt{prio})>\texttt{statistics}$ on the tag camera.motion indicates that
the camera’s motion detection data can only be exfiltrated in a pipeline that
includes one of the anonymization modules, and then the statistics module. The
boolean module forwards data along one input if the other input indicates a
condition has been met, while the prio [13] module is a technique for ensuring
data is analyzed in aggregate.

Pipeline permissions and exit policies are most useful in combination.
Pipeline permissions enable device functionality given a privacy tradeoff,
while exit policies restrict the conditions under which data can be
exfiltrated. When two policies conflict, the stricter one is enforced. For
example, Permission B in Fig. 6 conflicts with the exit policy J in Table 3.
The UI then alerts the user of this conflict to either remediate the existing
pipeline or accept a loss in functionality.

###  5.3 Enforcement Mechanism

The primary goal of the enforcement mechanism is simplicity for the vendors,
developers, and users of the framework. Vendors no longer bear the burden of
protecting user data because the data is locally stored and processed. Module
developers are agnostic of data labels such that they can write modular
functions without considering their global implications. Karl also uses a
simple, non-hierarchical data storage format, as opposed to accumulating
labels as data flows through the graph. At a high level, Karl creates a
modified version of the original dataflow graph to comply with user-defined
policies, then treats the modified graph as a declarative policy of permitted
dataflows, enforcing access control at the edges.

Karl determines how to modify the dataflow graph based on pipeline permissions
and exit policies. Karl denies the permissions explicitly denied by the user,
and the ones that conflict with exit policies. For each denied permission,
Karl either revokes network access or removes the edge to the device input.
These changes are applied in a secondary layer, ensuring the list of
permissions presented to the user remains the same. Denying one permission may
affect an allowed permission, such as if the path overlaps. In this case, Karl
can attempt to duplicate the overlapping graph such that the policies are
independent. If Karl were to identify the pipeline permissions in the modified
graph, they would include none of the denied permissions and as many of the
allowed permissions as possible, while following exit policies.

We treat the final dataflow graph as a declarative policy of permitted
dataflows between devices, modules, and the Internet. The resulting graph
nicely integrates vendor-defined functionality with user-defined policies.
Given the final graph, the sandbox enforces access control rules on the API
calls in Listing LABEL:lst:module-sdk. When a module pushes data, the sandbox
adds tags for the module’s output and the inputs on connected edges. When a
module reads data, it can only read the tags for its inputs. If the read is
along a stateless edge, the module cannot read data previously pushed to that
tag, a property aided by the ephemerality of modules. The sandbox allows
network accesses only to the domains specified for that module.

It could be interesting future work to explore how to enforce Karl’s policies
using IFC labels instead of non-hierarchical mandatory access control. Exit
policies as Boolean predicates would work nicely with Boolean labeling schemes
[51]. In this model, one might allow module developers to view the data
labels, though developers could then leak information through the labels. This
design choice could allow greater flexibility in enforcing policies based on
data provenance, at the cost of complexity [27, 21]. It remains interesting to
provide a theoretically-grounded approach to enforcing high-level policies in
a dataflow graph.

##  6 Karl Hub

Figure 8: System architecture.

The disaggregation of storage and compute from the IoT device allows
functionality to exist outside the device in a Karl hub. In contrast, typical
frameworks spread the functionality across local device state, companion apps,
and cloud services, in addition to the hub.

In this section, we describe how to deploy the hub on local network cloud
hardware managed by the user and discuss the tradeoffs. We then describe how
caching and data locality optimizations in the scheduler improve the
performance of Karl, particularly for resource-constrained environments.

###  6.1 Deployment

The Karl hub consists of sandboxes that execute Karl modules, and a controller
that schedules computation onto sandboxes (Fig. 8). The controller manages the
data store, and the webserver through which users configure and interact with
the smart home using Karl apps. All data and network accesses from a module
must go through the controller.

#### Local Deployment.

The recommended deployment uses local hardware to best moderate data
exfiltration outside the network boundary. To use the hub, users install a
program for the controller on a local server such as an old laptop. The
program can also come pre-installed on dedicated hardware. Then users install
sandbox programs on the same server (or others), making up the computational
capacity of the hub.

#### Cloud Deployment.

If resources are limited and scalability is an issue, it is possible to host
the hub on cloud hardware that is still managed by the user. The user can rent
a cloud server that meets the smart home’s computational demands at comparable
cost to IoT cloud subscriptions, with better privacy. In future work, we hope
to provide controller drivers for any backend that implements the data store
and module abstractions. This can include serverless platforms like AWS Lambda
or even community Karl sandboxes. Then the smart home can primarily leverage
local hardware but rollover to other hardware in the event of resource
constraints.

###  6.2 Scheduler

To improve performance in resource-constrained environments, we provide a
scheduler that optimizes for the smart home. In particular, writing stateful
cloud services with large data dependencies, such as person detection and
other ML tasks, as serverless modules results in high initialization and data
transfer times. In response, we implement data caching optimizations and
modify the scheduler to consider data locality to reduce the impact of these
modules on end user latency.

#### Cold-cache.

The cold-cache optimization mitigates the effect of large data dependencies on
increased network latencies. ML modules may contain large models, particularly
if they were offloaded to obviate IoT space constraints. Modules written in
interpreted languages may also contain large code dependencies. The sandbox
unpacks the files in a local filesystem cache, and mounts repeated modules
using aufs, an overlay filesystem, to not modify the root. To avoid sending
repeat dependencies, the controller tracks which modules it has spawned on
which sandboxes. If the sandbox has evicted the module, the controller takes
note and tries again.

#### Warm-cache.

The warm-cache optimization minimizes long intialization times by preemptively
launching slow modules and pausing until the first data or network access. The
sandbox then waits for the controller to spawn the same module and continues
executing at this point. An ML module might preemptively load its model in the
initialization phase, and handle an inference request as soon as the module is
re-spawned. Modules cannot be long-running services themselves, as they would
be unnecessarily stateful and leak data across invocations.

The scheduler manages a queue of modules that are ready to be spawned and
decides which sandbox to send each module to. Modules enter the queue based on
the schedule assigned to the module: when data is pushed to a particular tag,
at a regular interval (e.g., daily), or when manually triggered through the
webserver. Given a module, the scheduler selects a sandbox without an active
request, prioritizing those with the module cached. If a request does not
return after a certain timeout, the scheduler cancels the request and retries
on a different sandbox with an exponentially increasing timeout.

Though our scheduler prioritizes caching and availability, the scheduler could
consider many other factors to best utilize the resources available to it. One
example is to collect statistics about which sandboxes are fastest for
specific modules, and map modules to that sandbox. This is particularly
important on heterogeneous hardware. If Karl used a combination of local and
cloud hardware, the scheduler could account for privacy concerns on cloud
hardware, and data locality for latency. The scheduler could balance modules
based on whether the request needs real-time latency for a user interaction or
better hardware for a computationally expensive task.

##  7 Implementation

Module Name | LOC | Size | Language | Inputs | Outputs | Domains
---|---|---|---|---|---|---
boolean | 27 | 2.0MB | Rust | condition,input | output | -
firmware_update | 11 | 2.0MB | Rust | - | firmware | firmware.com
light_switch | 24 | 2.0MB | Rust | light_intent | state | -
person_detection | 46 | 981MB | Python | image | training_data,count | -
speech_to_intent | 75 | 38MB | Python | speech | weather_intent,light_intent | -
query | 36 | 2.1MB | Rust | image_data | result | -
set_true | 5 | 2.0MB | Rust | - | true | -
set_false | 5 | 2.0MB | Rust | - | false | -
statistics | 12 | 2.0MB | Rust | data | - | statistics.com
weather | 25 | 2.1MB | Rust | weather_intent | weather | weather.com
Table 4: Module implementations and specifications. The Rust modules are
$\approx$2.0MB because they all statically compile MUSL libc. The Python
modules are larger because Python is an interpreted language with more
dependencies.

We build a prototype of Karl in Rust, with SDKs in Rust and Python, in
$\sim$7000 LOC. The controller, sandboxes, devices, and modules communicate
over gRPC. The webserver passes tags and module names to Karl web apps via the
Handlebars templating language, authenticates each app with cookies, and
isolates app data from the Internet using Content Security Policy. Module
sandboxes use Firejail, an SUID program based on jails and namespaces, and a
Netfilter firewall.

We implemented 4 devices—a smart speaker, a light, a camera and an occupancy
sensor—and 10 modules ranging from ML to multi-device interactions, shown in
Table 4. We use Mask R-CNN [25] for person detection and Picovoice [41] for
speech-to-intent. We modeled devices as programs that push data to the hub at
a fixed interval. The camera pushes 156KB PNG images, and the microphone
pushes 172KB WAV audio files. We were able to put Karl camera firmware on a
hacked WyzeCam v2 [55], and a Karl smart speaker on a Raspberry Pi v4 with
hardware accessories.

We combined these devices and modules in two end-to-end applications to
highlight the range of functionality that Karl can express (Fig. 9). The first
application supports speech commands to a smart speaker that looks up the
weather or turns on a light. The second runs person detection when motion is
detected from the camera, sends training data to the Internet only when the
user is not home, and allows the camera to check for firmware updates.

##  8 Evaluation

We evaluate Karl to answer four questions:

  * •

Can Karl preserve existing device functionality in device-side logic, mobile
companion apps, and cloud services?

  * •

Can Karl enforce useful privacy policies that are difficult to enforce in
existing frameworks?

  * •

Does Karl provide reasonable latency for real-time interactions and ML
applications on local hardware?

  * •

How does hosting Karl on cloud hardware affect performance and cost?

We run the controller under Ubuntu 20.04 on a 10-year-old machine with two
4-core Xeon E5620 CPUs typical of discarded last-generation servers (CPUs
$8/pair on eBay) and 48GiB of RAM (mostly unused, as seen in Table 7). We run
a single sandbox and emulate the devices on the same server.

(a)

(b)

Figure 9: End-to-end applications with a speaker and a light (left), and an
occupancy sensor and two cameras (right).

###  8.1 Does Karl’s serverless programming model preserve device
functionality?

The applications we implemented in Fig. 9 demonstrate that Karl can preserve
the functionality of existing IoT devices in its programming model. We are
able to express device-side logic, companion apps, and cloud services using
local hardware under a single framework, unlike existing systems where IoT
devices are still tied to vendor infrastructure.

We found device-side logic to be modular and event-driven, fitting with Karl’s
modules and stateless edges. We designed edges to use common data types such
as image formats or JSON, such that even modules from different vendors could
be compatible. Otherwise, one could implement connector modules that convert
between formats. We used the boolean module to express control flow logic such
as conditional policies on whether the home was occupied or the time of day.

We designed Karl apps to match existing companion apps without a cloud
mediator. The light app lets the user adjust the light with low latency
because it communicates on the local network. The camera app lets the user
view a livestream on the user’s phone, preventing breaches where employees or
hackers of a vendor service see the feed [9, 53]. Storage is also controlled
by the user and kept on local hardware.

We implemented two ML modules that typically run as stateful cloud services:
speech-to-intent and person detection. When searching for ML implementations,
we were pleasantly surprised by the variety of open-source options [41, 42,
25, 24], depending on how users prioritize latency or accuracy. Though ML
typically runs on dedicated CPU and GPU servers in the cloud, our experiments
showed reasonable latency on our hub’s 10-year-old CPU. In general, we believe
that ML applications previously thought to be too costly for local hardware
will be cheaper to support in the future, as commodity hardware vendors are
implementing ML acceleration, e.g. in Intel’s integrated GPUs and Apple’s
Neural Engine.

###  8.2 Can Karl enforce useful privacy policies?

To determine what is considered “useful", we analyzed Karl in terms of
existing studies on smart home users. Zheng et al. [56] suggests that users
care strongly about audio-visual data, would prefer sharing data in aggregate,
and prioritize functionality over privacy. Dixon et al. [16] recommends time-
based access control and extra sensitive devices as security primitives, and a
central management layer. The Karl hub can enforce these properties with exit
policies on tags for audio-visual or sensitive data, conditions based on
anonymization or boolean modules for time-based policies, and a framework that
that tightly integrates functionality and privacy.

We validated Karl’s generality by implementing all the privacy properties in
Fig. 5. Existing research frameworks either cannot express or enforce all
these policies (§3). Hubs integrate local devices, but do not account for data
handled in companion apps and vendor services [54, 21, 52]. Another reason is
the granularity at which frameworks define policies. Low-level IPCs lack
semantic meaning [31], while high-level app descriptions do not directly match
functionality [52].

Next, we discuss the enforcement mechanism and how it reflects the users’
privacy expectations. In many cases, denying a pipeline permission does not
affect other pipelines, such as when we denied the speaker from turning on the
light bulb in Fig. 9. In other cases, it affects other permissions that the
user has allowed, leading to unintended side effects. When we denied the
occupancy sensor from sending data to statistics.com, Karl revoked network
access from the statistics module, and the camera was no longer able to share
training data. This side effect makes sense—when we decided not to leak
metadata about our occupancy, the boolean module could no longer properly
anonymize training data, so the camera pipeline was also denied. The UI
indicates when pipeline permissions conflict with exit policies and each other
to convey side effects to users, though user studies should be performed to
improve the design. Karl still provides the foundation on which IoT hubs can
build to express and enforce comprehensive privacy policies on modular
application semantics.

###  8.3 How is performance on local hardware?

Figure 10: End-to-end latencies on local vs. cloud deployment with warm-cache optimization. AVX2 support on cloud hardware offsets WAN latencies in PersonDet (Pipeline III). |  Pipeline | Real-time? | ML?
---|---|---|---
LivestreamOn (I) |  set_true $\rightarrow$ camera.livestream | yes | no
SpeechLight (II) |  speaker.speech_command $\rightarrow$ speech_to_intent $\rightarrow$ light_switch $\rightarrow$ light.state | yes | yes
PersonDet (III) |  camera.motion $\rightarrow$ person_detection ($+$ occupancy_sensor.at_home) $\rightarrow$ boolean $\rightarrow$ statistics $\rightarrow$ statistics.com | no | yes
Table 5: Evaluated pipelines for end-to-end latency.

Fig. 10 demonstrates that Karl has reasonable end-to-end latency for real-time
user interaction and computational tasks in a resource-constrained
environment. We evaluated combinations of these two types of requests using
the pipelines in Table 5. We measured end-to-end latency as the time from when
a user interacts with system to when the user can observe its intended effect.
Latencies are the average of 5 trials, after warmup. All performance
optimizations are enabled.

We tested two real-time user interactions and observed they are within a
reasonable human response time. We turned on a camera livestream through a
Karl app in 2ms, and turned on a light through a speech command in 236ms. In
comparison, we asked an Amazon Alexa the weather as a simple experiment and
only received a response $2.1$s after finishing the command. Device usability
relies on real-time interactions, an area where Karl’s local computation
excels.

Computational tasks can be constrained by user hardware, though they do not
necessarily require real-time latency. Person detection and post-processing
took $8.1$s, bottlenecked by the lack of AVX2 instructions in our hardware (it
took $3.1$s on cloud hardware with AVX2). Unless the Karl hub is fully
saturated, we do not expect occasionally compute-heavy tasks like this to
significantly impact user experience.

| Baseline | Cold-cache | Warm-cache
---|---|---|---
LivestreamOn | 64ms | 54ms (16%) | 2ms (97%)
SpeechLight | 740ms | 515ms (30%) | 236ms (68%)
PersonDet | 17.0s | 11.1s (34%) | 8.1s (53%)
(a) Local deployment.

| Baseline | Cold-cache | Warm-cache
---|---|---|---
LivestreamOn | 126ms | 99ms (21%) | 18ms (86%)
SpeechLight | 1110ms | 894ms (19%) | 488ms (56%)
PersonDet | 10.1s | 5.2s (49%) | 3.1s (69%)
(b) Cloud deployment.

Table 6: Effect of caching optimizations on end-to-end latencies of the three
pipelines in Table 5, where the baseline is Karl with no optimizations. In
parentheses, percent improvement over baseline.

Table 11(a) demonstrates how our caching optimizations improved performance.
The cold-cache optimization most benefited Pipeline III by eliminating the
time to resend ML data dependencies and Python libraries over the network. The
size of the person_detection module was $490\times$ the size of the set_true
module in Pipeline I, which benefited the least. The warm-cache optimization
most benefited Pipeline I, which had long initialization times for mounting
the filesystem and establishing a sandbox, relative to the remaining
execution. In Pipeline III, warm-cache eliminated the pre-processing time of
loading a model, reducing latency by $3.0$s.

###  8.4 How does hosting Karl in the cloud affect performance and cost?

| Controller | Host | Host
---|---|---|---
| (Max Mem.) | (Max Mem.) | CPU%
Baseline | 8.7 MB | 8.6 MB | 0%
I (warm) | 14.7 MB | 15.3 MB | 3%
I (cold) | 14.5 MB | 13.0 MB | 3%
II (warm) | 1.25 GB | 938 MB | 9%
II (cold) | 1.26 GB | 846 MB | 9%
III (warm) | 3.03 GB | 2.04 GB | 92%
III (cold) | 3.03 GB | 2.02 GB | 92%
Table 7: Resource utilization metrics for each pipeline running alone on cloud
hardware. The baseline is measured before registering any devices. Most memory
usage comes from writing data dependencies from the network request to disk in
the initial cold cache request. The warm cache keeps active modules in memory.

We compared the network latency of the same pipelines using Karl deployed on
user-managed cloud hardware. We used an m510 CloudLab server with an Intel
Xeon D-1548 processor with 8 cores and 64GiB RAM. We observed 16ms more
latency from Pipeline I, which sent a request to the camera over the WAN with
larger and more variable latencies. Module data accesses still went over the
same LAN as the hub.

Another aspect of performance is the hardware. Notably, the processor on the
CloudLab server supports AVX2. If the user has particular applications they
want to run that rely on special hardware for performance, cloud platforms
could give them flexibility in selecting exactly the resources they need.

A back-of-the-envelope calculation finds that we can deploy Karl in AWS for
$14.80/month, which is comparable to IoT subscriptions like Ring Protect for
$10/month with greater privacy. Most IoT traffic is frequent and low-bandwidth
[38] similar to Pipeline I, so we assume the bottlenecks to involve ML. Given
the CPU and memory usage statistics of each pipeline (Table 7), we select an
instance that can handle Pipeline III. Even if multiple cameras handle 100
requests/day, these tasks are hardly enough to fully utilize a sandbox if they
rarely overlap. On AWS, a t2.medium reserved instance with 2 vCPUs and 4.0 GiB
RAM costs $14/month, and 10 GB of EBS storage is $0.80/month. We hope to
reduce these costs in future work by leveraging serverless cloud platforms for
Karl sandboxes instead [40].

##  9 Discussion

In this section, we discuss some of the more practical concerns about adopting
Karl.

#### Administrative benefits of local hosting.

We argue the convenience of cloud hosting is overstated. The Karl hub has the
same uptime as the home router on which it depends for connectivity, and is
not affected by cloud outages [5, 1, 15, 39] Another issue Karl addresses is
the heterogeneity of individual vendors [33, 22, 34], who can focus on
building software and hardware rather than rebuilding the same infrastructure.
Karl can integrate security best practices such as multi-factor authentication
and encrypted network protocols directly into the framework, and durability
measures such as encrypted backups on cloud or decentralized storage [37, 7].

#### Vendor incentives to adopt Karl.

Vendors can provide users lower latency for interactive applications and real-
time processing in bandwidth-limited wide-area networks, similar to edge
computing [45]. Smaller vendors in particular would benefit from lower upfront
costs without having to recoup the costs of hosting cloud infrastructure, and
not having to invest in complying with stricter legislation trends.

#### Business model other than data monetization.

We imagine a world in which users are willing to pay a privacy premium for
just hardware devices and software modules, without sacrificing their data. It
is reasonable to run proprietary software on user hardware, as in the mobile
app market. Karl also does not prevent vendors from collecting data, as long
as they do it transparently and with justification. Karl is particularly
compatible with client-side anonymization techniques [29, 13, 18, 10, 32].

#### Privacy legislation trends.

There has been a trend towards stricter privacy laws such as GDPR [14, 36].
Karl automatically ensures privacy through local hosting so vendors without
the security expertise to comply can focus on building software and hardware.
In general, privacy laws need to strike a balance between what protects user
data most and what is practical for vendors to implement. We hope Karl and
future research can influence legislation by demonstrating the practicality of
more privacy-friendly approaches.

#### Transition strategy.

The easiest way to transition to Karl would be for existing frameworks such as
SmartThings and Home Assistant to adopt its ideas. Karl’s event-driven
programming model shares many similarities, but the difference is how these
frameworks leverage local hosting and modularity to provide privacy
guarantees. Existing frameworks should be more restrictive about their concept
of a “module”, and its data and network privileges. They should determine all
pathways for sensitive data to be leaked, including the companion apps, cloud
services, and multi-device interactions of modern devices. Adopting these
ideas can provide the foundation for privacy policies such as pipeline
permissions and exit policies that can be enforced and expressed by the user.

##  10 Conclusion

IoT devices that belong to end users should not depend on vendor-maintained
infrastructure to operate. Karl provides a better model that prioritizes
privacy while preserving the functionality of modern devices: IoT devices
outsource computation and storage to user-controlled hardware, on which Karl
sandboxes software modules from different vendors. With Karl, vendors
extricate themselves from hosting cloud infrastructure, and users can
unilaterally enforce coherent security across devices and modules. Karl
introduces two security mechanisms: _pipeline permissions_ that permit device
data to be shared given some justification and _exit policies_ that block
flows unless specific conditions are met. We demonstrate Karl’s viability
through several IoT applications with comparable performance and much greater
privacy.

## References

  * [1] Noor Al-Sibai.  Amazon outage shuts down iot vacuums, doorbells, fridges, even home locks, Dec 2021.
  * [2] Omar Alrawi, Chaz Lever, Manos Antonakakis, and Fabian Monrose.  Sok: Security evaluation of home-based iot deployments.  In 2019 IEEE symposium on security and privacy (sp), pages 1362–1380. IEEE, 2019.
  * [3] Manos Antonakakis, Tim April, Michael Bailey, Matt Bernhard, Elie Bursztein, Jaime Cochran, Zakir Durumeric, J Alex Halderman, Luca Invernizzi, Michalis Kallitsis, et al.  Understanding the mirai botnet.  In 26th $\\{$USENIX$\\}$ security symposium ($\\{$USENIX$\\}$ Security 17), pages 1093–1110, 2017.
  * [4] Apple Home.  https://www.apple.com/ios/home/, 2021.
  * [5] Aws: Amazon web outage breaks vacuums and doorbells, Nov 2020.
  * [6] AWS Lambda.  https://aws.amazon.com/lambda/, 2022.
  * [7] Juan Benet.  Ipfs-content addressed, versioned, p2p file system.  arXiv preprint arXiv:1407.3561, 2014.
  * [8] Ring video doorbell pro under the scope, Nov 2019.
  * [9] Dalvin Brown.  Amazon’s ring fires four employees for snooping on customers’ doorbell camera video feeds, Jan 2020.
  * [10] Lukas Burkhalter, Nicolas Küchler, Alexander Viand, Hossein Shafagh, and Anwar Hithnawi.  Zeph: Cryptographic enforcement of end-to-end data privacy.  In 15th $\\{$USENIX$\\}$ Symposium on Operating Systems Design and Implementation ($\\{$OSDI$\\}$ 21), pages 387–404, 2021.
  * [11] Giovanni Campagna, Rakesh Ramesh, Silei Xu, Michael Fischer, and Monica S Lam.  Almond: The architecture of an open, crowdsourced, privacy-preserving, programmable virtual assistant.  In Proceedings of the 26th International Conference on World Wide Web, pages 341–350, 2017.
  * [12] Z Berkay Celik, Leonardo Babun, Amit Kumar Sikder, Hidayet Aksu, Gang Tan, Patrick McDaniel, and A Selcuk Uluagac.  Sensitive information tracking in commodity iot.  In 27th $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 18), pages 1687–1704, 2018.
  * [13] Henry Corrigan-Gibbs and Dan Boneh.  Prio: Private, robust, and scalable computation of aggregate statistics.  In 14th $\\{$USENIX$\\}$ Symposium on Networked Systems Design and Implementation ($\\{$NSDI$\\}$ 17), pages 259–282, 2017.
  * [14] Council of European Union.  Council regulation (EU) no 269/2014, 2014.  http://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1416170084502&uri=CELEX:32014R0269.
  * [15] Mehdi Daoudi.  What 2019’s ’summer of outages’ means for iot, Jan 2020.
  * [16] Colin Dixon, Ratul Mahajan, Sharad Agarwal, AJ Brush, Bongshin Lee, Stefan Saroiu, and Paramvir Bahl.  An operating system for the home.  In 9th $\\{$USENIX$\\}$ Symposium on Networked Systems Design and Implementation ($\\{$NSDI$\\}$ 12), pages 337–352, 2012.
  * [17] Julia Brande Earp, Annie I Antón, Lynda Aiman-Smith, and William H Stufflebeam.  Examining internet privacy policies within the context of user privacy values.  IEEE Transactions on Engineering Management, 52(2):227–237, 2005\.
  * [18] Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova.  Rappor: Randomized aggregatable privacy-preserving ordinal response.  In Proceedings of the 2014 ACM SIGSAC conference on computer and communications security, pages 1054–1067, 2014.
  * [19] Adrienne Porter Felt, Elizabeth Ha, Serge Egelman, Ariel Haney, Erika Chin, and David Wagner.  Android permissions: User attention, comprehension, and behavior.  In Proceedings of the eighth symposium on usable privacy and security, pages 1–14, 2012.
  * [20] Earlence Fernandes, Jaeyeon Jung, and Atul Prakash.  Security analysis of emerging smart home applications.  In 2016 IEEE symposium on security and privacy (SP), pages 636–654. IEEE, 2016.
  * [21] Earlence Fernandes, Justin Paupore, Amir Rahmati, Daniel Simionato, Mauro Conti, and Atul Prakash.  Flowfence: Practical data protection for emerging iot application frameworks.  In 25th $\\{$USENIX$\\}$ security symposium ($\\{$USENIX$\\}$ Security 16), pages 531–548, 2016.
  * [22] Cesare Garlati.  Owlet baby wi-fi monitor “worst iot security of 2016”, Oct 2016.
  * [23] Andy Greenberg.  Hackers found a (not-so-easy) way to make the amazon echo a spy bug, Aug 2018.
  * [24] Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, et al.  Deep speech: Scaling up end-to-end speech recognition.  arXiv preprint arXiv:1412.5567, 2014.
  * [25] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick.  Mask R-CNN.  CoRR, abs/1703.06870, 2017.
  * [26] Home Assistant.  https://www.home-assistant.io/, 2021.
  * [27] Tyler Hunt, Zhiting Zhu, Yuanzhong Xu, Simon Peter, and Emmett Witchel.  Ryoan: A distributed sandbox for untrusted computation on secret data.  ACM Transactions on Computer Systems (TOCS), 35(4):1–32, 2018.
  * [28] IFTTT.  https://ifttt.com/, 2021.
  * [29] Suman Jana, David Molnar, Alexander Moshchuk, Alan Dunn, Benjamin Livshits, Helen J Wang, and Eyal Ofek.  Enabling fine-grained permissions for augmented reality applications with recognizers.  In 22nd $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 13), pages 415–430, 2013.
  * [30] Carlos Jensen and Colin Potts.  Privacy policies as decision-making tools: an evaluation of online privacy notices.  In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pages 471–478, 2004.
  * [31] Yunhan Jack Jia, Qi Alfred Chen, Shiqi Wang, Amir Rahmati, Earlence Fernandes, Zhuoqing Morley Mao, Atul Prakash, and SJ Unviersity.  Contexiot: Towards providing contextual integrity to appified iot platforms.  In NDSS, volume 2, pages 2–2, 2017.
  * [32] Ronny Ko and James Mickens.  Deadbolt: Securing iot deployments.  In Proceedings of the Applied Networking Research Workshop, pages 50–57, 2018.
  * [33] Brian Krebs.  Ubiquiti all but confirms breach response iniquity, Apr 2021.
  * [34] Selena Larson.  Fda confirms that st. jude’s cardiac devices can be hacked, Jan 2017.
  * [35] Sanghak Lee, Jiwon Choi, Jihun Kim, Beumjin Cho, Sangho Lee, Hanjun Kim, and Jong Kim.  Fact: Functionality-centric access control system for iot programming frameworks.  In Proceedings of the 22nd ACM on Symposium on Access Control Models and Technologies, pages 43–54, 2017.
  * [36] California Legislature.  The california consumer privacy act of 2018, Jun 2018.
  * [37] Ali José Mashtizadeh, Andrea Bittau, Yifeng Frank Huang, and David Mazieres.  Replication, history, and grafting in the ori file system.  In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, pages 151–166, 2013.
  * [38] TJ OConnor, Reham Mohamed, Markus Miettinen, William Enck, Bradley Reaves, and Ahmad-Reza Sadeghi.  Homesnitch: behavior transparency and control for smart home iot devices.  In Proceedings of the 12th Conference on Security and Privacy in Wireless and Mobile Networks, pages 128–138, 2019.
  * [39] Charlie Osborne.  Over 60 million wearable, fitness tracking records exposed via unsecured database, Sep 2021.
  * [40] Shoumik Palkar and Matei Zaharia.  DIY hosting for online privacy.  In Proceedings of the 16th ACM Workshop on Hot Topics in Networks, pages 1–7, 2017.
  * [41] picovoice.  https://github.com/Picovoice/picovoice, 2021.
  * [42] Rhasspy.  https://rhasspy.readthedocs.io/en/latest/, 2021.
  * [43] Chris Roberts.  Parler was hacked on wordpress, the internet’s biggest platform. is everyone at risk?, Jan 2021.
  * [44] Franziska Roesner, Tadayoshi Kohno, Alexander Moshchuk, Bryan Parno, Helen J Wang, and Crispin Cowan.  User-driven access control: Rethinking permission granting in modern operating systems.  In 2012 IEEE Symposium on Security and Privacy, pages 224–238. IEEE, 2012.
  * [45] Mahadev Satyanarayanan.  The emergence of edge computing.  Computer, 50(1):30–39, 2017.
  * [46] Paulus Schoutsen.  Almond & ada: privacy-focused voice assistant, Nov 2019.
  * [47] Ashley Sears.  ’felt so violated:’ milwaukee couple warns hackers are outsmarting smart homes, Sep 2019.
  * [48] Simon Shillaker and Peter Pietzuch.  Faasm: lightweight isolation for efficient stateful serverless computing.  In 2020 $\\{$USENIX$\\}$ Annual Technical Conference ($\\{$USENIX$\\}$$\\{$ATC$\\}$ 20), pages 419–433, 2020.
  * [49] SmartThings.  https://www.smartthings.com/, 2021.
  * [50] Vikram Sreekanti, Chenggang Wu, Xiayue Charles Lin, Johann Schleier-Smith, Jose M Faleiro, Joseph E Gonzalez, Joseph M Hellerstein, and Alexey Tumanov.  Cloudburst: Stateful functions-as-a-service.  arXiv preprint arXiv:2001.04592, 2020.
  * [51] Deian Stefan, Alejandro Russo, David Mazières, and John C Mitchell.  Disjunction category labels.  In Nordic conference on secure IT systems, pages 223–239. Springer, 2011.
  * [52] Yuan Tian, Nan Zhang, Yueh-Hsun Lin, XiaoFeng Wang, Blase Ur, Xianzheng Guo, and Patrick Tague.  Smartauth: User-centered authorization for the internet of things.  In 26th $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 17), pages 361–378, 2017.
  * [53] William Turton.  Surveillance cameras at tesla, many others breached: Report, Mar 2021\.
  * [54] Qi Wang, Wajih Ul Hassan, Adam Bates, and Carl Gunter.  Fear and logging in the internet of things.  In Network and Distributed Systems Symposium, 2018.
  * [55] Xiaomi DaFang Hacks / XiaoFang 1S / Wyzecam V2 / Wyzecam Pan / Other T20 Devices.  https://github.com/EliasKotlyar/Xiaomi-Dafang-Hacks, 2021.
  * [56] Serena Zheng, Noah Apthorpe, Marshini Chetty, and Nick Feamster.  User perceptions of smart home iot privacy.  Proceedings of the ACM on Human-Computer Interaction, 2(CSCW):1–20, 2018.