# Software Update Practices on Smart Home IoT Devices

Vijay Prakash  New York UniversityNew YorkUSA<EMAIL_ADDRESS>,  Sicheng
Xie  New York UniversityNew YorkUSA<EMAIL_ADDRESS>and  Danny Yuxing Huang  New
York UniversityNew YorkUSA<EMAIL_ADDRESS>
(2018)

###### Abstract.

Smart home IoT devices are known to be breeding grounds for security and
privacy vulnerabilities. Although some IoT vendors deploy updates, the update
process is mostly opaque to researchers. It is unclear what software
components are on devices, whether and when these components are updated, and
how vulnerabilities change alongside the updates. This opaqueness makes it
difficult to understand the security of software supply chains of IoT devices.

To understand the software update practices on IoT devices, we leverage IoT
Inspector’s dataset of network traffic from real-world IoT devices. We analyze
the User Agent strings from plain-text HTTP connections. We focus on four
software components included in User Agents: cURL, Wget, OkHttp, and python-
requests. By keeping track of what kinds of devices have which of these
components at what versions, we find that many IoT devices potentially used
outdated and vulnerable versions of these components—based on the User
Agents—even though less vulnerable, more updated versions were available; and
that the rollout of updates tends to be slow for some IoT devices.

IoT; supply chain; updates

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation emai; June 03–05, 2018; Woodstock, NY††price:
15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs: Security and privacy Mobile and
wireless security††ccs: General and reference Empirical studies

##  1\. Introduction

Smart home technologies, also known as smart devices or Internet-of-Things
(IoT) devices, are gaining popularity, such as smart TVs, speakers, cameras,
and medical devices, yet they are breeding grounds for security and privacy
threats (Antonakakis et al., 2017; Kumar et al., 2019), using vulnerable
software components (TLS, 2022) and putting the users and other hosts on the
network at risk.

To mitigate these risks, like a software, many IoT devices offer updates that
can be installed automatically or manually by users. However, this update
process remains largely opaque; it is unclear what software components are on
devices, whether/how any software updates have occurred on these software
components, and how vulnerabilities change alongside the updates across a
large number, variety, and the long tail of IoT devices in smart homes (IoT,
2021). This opaqueness makes it hard to keep track of the software bill-of-
materials (SBOMs) across devices over time and understand the software supply
chain of smart home IoT devices in general.

One of the reasons for this knowledge gap is the lack of scale in many lab-
based analyses of IoT devices. To understand the software components on IoT
devices and the update behaviors, researchers often analyze the firmware
binaries. However, commercial smart home IoT devices tend to use proprietary
and/or protected software components (such as the firmware and libraries) that
are difficult to extract and reverse-engineer (Mohajeri Moghaddam et al.,
2019). Furthermore, researchers often need the physical IoT devices for their
analyses. The number of devices that can be studied is often constrained by
time and budget; one of the largest sample of IoT devices studied in the lab
is about 120 devices (Ren et al., 2019; Paracha et al., 2021) and does not
cover many devices in the long tail (IoT, 2021). As such, it is difficult for
researchers to identify whether and how IoT devices update to patch
vulnerabilities over time.

Figure 1. Software through vulnerability cycle. In this study, we are focusing
on the updates, step 4, 5 and 6.

Figure 1 shows software through vulnerability life cycle: 1) A software is
released; 2) Security researchers find vulnerabilities; 3) Researchers
ethically notify vendors about the vulnerabilities privately and give a grace
period of 90 days (current standard) to fix those issues and release updates;
4) Vendor publicly discloses the vulnerability along with updates containing
fixes; 5) In the ideal case, users download the updates and update the
software making it secure against the exploits; 6) In the worst case, users
don’t install the update and software stays vulnerable. Finally the cycle
continues starting from vulnerability research again after both the 5th and
6th stage of the cycle.

##### Objectives

Our goal is to understand software update practices on IoT devices in real-
world smart homes: what software components and versions are present on
devices at a particular time, as well as when the versions and the associated
vulnerabilities have changed over time (i.e., likely updates). This knowledge
will shed light on the update practices across a large number and variety of
IoT devices and vendors, thus paving the way for deeper empirical
understanding of IoT software supply chain security.

##### Method

To achieve this goal, we conduct the first known longitudinal analysis of IoT
software components in real-world smart homes and provide an initial
understanding of update practices in the IoT ecosystem. In particular, we
leverage IoT Inspector’s dataset of network traffic crowdsourced from smart
homes around the world (Huang et al., 2020). The dataset includes the likely
identities of devices, along with any User-Agent strings extracted from plain-
text HTTP headers sent by these devices. The User Agents indicate potential
use of certain software components and their versions. We would like to note
that devices whose HTTP traffic we have captured do not use plain HTTP for the
majority of communication, but they do send them out for some reason with
empty payload, possibly to test a connection or discovery purposes (based on
our lab studies). We have not decrypted the HTTPS connections while collecting
data for this study.

In this paper, we analyze the versions of four different software components
that appear in the User Agent strings: cURL (which is both a library, libcurl,
and a binary), Wget (a binary), OkHttp library, and Requests (Python’s)
library. We observe these components in the User Agent strings on 23,837
devices (including 359 what looks like IoT devices) across 4,562 real-world
smart homes between Apr 2019 and Oct 2021.

##### Findings.

We show that update practices in the IoT ecosystem are different from
(sometimes worse than) findings from general purpose computing devices. In
particular, none of the IoT devices in our dataset included the latest
versions of the four software components in the User Agent, even when the
latest versions were available at the time IoT Inspector captured the data. In
some cases, the lower versions included critical CVEs that could have been
fixed if the IoT devices had been using the latest versions at the time.
Furthermore, we find that vendor deploy updates in rolling fashion, and
oftentimes rolled out updates are not the latest. In other words, when vendors
deploy updates, they do not always update the software component to the latest
versions, which means oftentimes devices are left in vulnerable state even
after end user install the update.

These findings paint a grim picture of the slow (and sometimes the lack of)
software update practices on certain IoT devices. Such update practices could
be a result of users not promptly installing updates, IoT vendors not
deploying updates in time, or both—it is an open question what is the cause
for our observation. It is our hope that this study will offer the first
evidence on the update practices on IoT devices, highlight the associated
software supply chain risks, and provide the impetus for more secure practices
on users, IoT vendors, and regulators.

##  2\. Related Work

##### Software updates.

In the last couple of decades, there has been a push from academia and
industry to improve the software update hygiene for all kinds of software
because old software has been correlated with compromise (Bilge et al., 2017;
Khan et al., 2012), so it is in the best interest of everyone’s security and
from economical standpoint to use the latest version of software (DeKoven et
al., 2019; Sarabi et al., 2017; Arbaugh et al., 2000; Clark et al., 2014;
Morgner et al., 2020). Propelled by this push, major software vendors have
adopted the practice of deploying frequent updates, which improved the vendor
update deployment practice. On the other hand, end users are advised to follow
“Best Practices”, which includes updating software as soon as updates are
available if users can (Bilge et al., 2017; Khan et al., 2012; Sarabi et al.,
2017). For software like operating systems (OS) and applications running on
them, previous studies have shown that user update installation practice can
be modeled with either geometric or exponential distribution, which means
majority of the users update the software after new release, with some users
taking longer duration to install the updates (Sarabi et al., 2017; Rescorla,
2003; Ramos, 2006; Kandek, 2009).

Updating software is considered to be a part of ”Best Practices” to increase
security by shrinking the vulnerable state of hosts, (Reeder et al., 2017;
DeKoven et al., 2019) recommend updates. Khan et al. (Khan et al., 2012) found
a positive correlation between infection indicators and a lack of regular
updating practice, (Bilge et al., 2017) showed correlation in devices not
updating and being compromised at some point of time. Sarabi et al.(Sarabi et
al., 2017) found that frequent discovery of vulnerabilities in a software
limits the benefit of providing faster updates. DeKoven et al. (DeKoven et
al., 2019) measure the correlation between “Best Practices” and impact on
security risk.

##### Four factors for update.

One way to define security of a host is how vulnerable it is; the less
vulnerability a host has, the more secure it likely is. From the software
update perspective, there are four factors that decide the vulnerable state of
a host: 1) how quickly users install the updates; 2) how quickly vendors
deploy the patch; 3) how vendors deploy the patch; and 4) how frequently
vulnerabilities are found (Sarabi et al., 2017). These combined together tell
about update practice. So far studies have looked at all the four factors for
software that are used for computing in general (i.e. mobiles, computers,
tablets, etc.) (Alhazmi et al., 2007; Clark et al., 2014; Arbaugh et al.,
2000; Cavusoglu et al., 2005; Alhazmi and Malaiya, 2005; Arora et al., 2004;
Neuhaus et al., 2007; Rescorla, 2005; Shahzad et al., 2012; Durumeric et al.,
2014; Nappa et al., 2015; Ramos, 2006; Rescorla, 2003; Yilek et al., 2009;
Sarabi et al., 2017).

There have been studies about all the factors affecting the vulnerability
state of a host. These studies have been conducted for both clients side
(general purpose computing devices) and server side hosts. For general purpose
computing devices, there is (Sarabi et al., 2017; DeKoven et al., 2019) on
user update behaviors using various software; (Nappa et al., 2015; Arbaugh et
al., 2000; Clark et al., 2014; Alhazmi and Malaiya, 2005; Alhazmi et al.,
2007; Arora et al., 2004; Neuhaus et al., 2007; Rescorla, 2005; Shahzad et
al., 2012) on vendor update deployment and vulnerability disclosure;
(Gkantsidis et al., 2006; Frei et al., 2009; Duebendorfer and Frei, 2009) on
mechanism of deploying patches where they suggest silent updates are the most
effective. On servers side hosts update behaviors there are (Ramos, 2006;
Kandek, 2009; Rescorla, 2003; Durumeric et al., 2014; Yilek et al., 2009),
where they measure patch installation delays after major security incidents
and routine patch releases after vulnerability disclosures.

##### Update practice for IoT.

Specific to IoT devices, Yousefnezhad et al.(Yousefnezhad et al., 2020) has
done a comprehensive survey of IoT product lifecycle, including vulnerability
management and software updates. There have been numerous study about IoT
update infrastructure, i.e., how to deliver updates to IoT devices
logistically (Yohan and Lo, 2018; Kim et al., 2018; Zhang et al., 2016; Langiu
et al., 2019), and securely (Bettayeb et al., 2019). It has also been found
that user awareness about updating devices has improved — IoT device users
prefer to buy products that guarantees updates (Morgner et al., 2020).

So far we are not aware of any study about the current update practice in IoT
ecosystem. The case for IoT devices is different from major software vendors
because of the sheer amount of devices and vendors available in the market
(IoT, 2021). Another possible reason for the differentiation could be that
vendor nudges to update software could not reach IoT device users, unlike
computing devices if users are actively using devices they see the update
nudge on UI. Because some IoT devices can only be interacted with companion
apps, and users might forget to use the apps as these devices might be always
on without user interaction. Also, unlike few major software vendors for
personal computing, IoT vendors are fragmented and too many (IoT, 2021). All
this makes it difficult to apply the results of previous studies to the IoT
ecosystem.

##  3\. Method

###  3.1. Inferring software components from User Agents

To understand software update practices on real-world IoT devices, we use IoT
Inspector’s (Huang et al., 2020) dataset of network traffic, crowdsourced from
global smart home users. The dataset includes names and manufacturers of
devices, along with various network header data, including the User Agent
strings extracted from plain-text HTTP connections. Note that IoT Inspector
does not capture the contents of the HTTP traffic; nor does it collect
personally identifiable information (Huang et al., 2020). We look at a subset
of this data, which consists of 4,562 different smart homes containing 23,837
unique devices (including IoT and general purpose computing) from Apr 9, 2019
to Oct 12, 2021.

The captured User Agent strings offer indications for software components that
devices have likely used. We focus on four software components used as User
Agents: cURL (”cURL”, 2022), Wget (”Wget”, 2022), Python Requests (Requests”,
2022), and OkHttp (”OkHttp”, 2022). We picked these because they are the most
popular (by the number of devices) User Agents that are not browsers. This
information is by no means the ground truth for what software components
devices actually used (since HTTP clients can, in theory, put anything into
the User Agent field). Still, we assume that devices’ HTTP clients are honest
about being non-browsers. There is evidence for non-truthful User Agents for
browsers (UA-, 2022), but we are unaware of any evidence of non-truthful User
Agents for non-browsers like cURL, Wget, Requests, and OkHttp. As such,
throughout this paper, we use the User Agent as a proxy for the actual
software components.

IoT Inspector captures the HTTP User-Agent header seen in the network traffic
from uniquely identified devices with a timestamp of when it was seen. Devices
whose traffic is captured could include general purpose computing devices
(computers, phones, etc.) and IoT devices. We distinguish the device
categories with the Fingerbank (”Fingerbank”, 2022) APIs (which identify
devices based on the MAC address and destination hosts contacted over a
proprietary ML model).

To find out how old and vulnerable versions of these four software are used,
we manually compile a list of versions released with dates from their websites
or code bases. Each version is assigned a release number from 0 (first
release) to N (last release). We also compile a table of vulnerabilities each
version of these four software have. Vulnerabilities in each version are
either available on their website or we compile it by searching names of each
software on the National Vulnerability Database (NVD) (NVD, 2022) and use the
CVE number and Common Platform Enumeration (CPE) to map them to different
versions.

###  3.2. Metrics for understanding update practices

To get the understanding of update practices, from the collected data where
HTTP user agent is one of the four software, and compilation of four software
version release dates and vulnerability each version have, we curate
statistical metrics for all the unique device and user agents combination.
This includes following:

  * •

What software is used as user agent and its used version

  * •

First and last seen timestamp in the IoT Inspector dataset

  * •

At the last seen time, the latest available version of that software

  * •

At the last seen time, the number of versions and days (or months) behind this
used user agent version is from this latest available version at this time

  * •

At the last seen time, the number of vulnerabilities that could have been
avoided by using the latest version. We explain how this is calculated later
in the next subsection 3.3.

For our analysis to gain perspective of update practices we rely on three
metrics retroactively:

  1. (1)

number of avoidable vulnerabilities, having vulnerabilities directly
contribute to the vulnerable state of a host adversely

  2. (2)

number of versions behind

  3. (3)

and number of days behind (directly correlated with number of versions
behind), they both show how quickly hosts are updated which involves both user
promptness in installing updates and vendors deploying updates.

From our data set we can’t distinguish if users are not installing updates
promptly or vendors are not deploying updates, but we can tell about the
updates practices comprising both with these three metrics.

###  3.3. How these metrics are calculated

Versions behind for a user agent when it’s seen in our data is calculated by
subtracting the release number of the used version from release number of most
recent available version at this seen time. To calculate the number of
avoidable vulnerabilities, we take the difference between the set of all the
vulnerabilities in the used version and the set of vulnerabilities in the most
recent version, which tells us vulnerabilities that were in the used version
but not in the latest version, i.e., they were fixed in the latest version.
Let’s say, X be the set of vulnerabilities in the used version and Y be the
set of vulnerabilities in the latest available version. Number of avoidable
vulnerabilities would be (X - Y), whereas this - (subtraction) represents the
difference of set operation.

Figure 2. Distribution of different metrics for all software components in our
data. Figure 3. Number of avoidable vulnerabilities in cURL against number of
versions behind, and CDF distribution of devices for number of versions
behind.

For example, let’s suppose cURL version 7.20.0, which is release # 112,
released on 2010-02-09, and has 10 vulnerabilities as of today. Let’s also
suppose we see a device with the user agent cURL 7.20.0 on 2020-02-09 in the
dataset. Latest available version of cURL on 2020-02-09 is version 7.68.0,
which is release # 187, released on 2020-01-08, and has 8 vulnerabilities as
of today. The number of versions behind for this seen version is 75 and the
number of days behind is 3620 (9 years and 11 months). For the sake of this
example, let’s assume 8 of the vulnerabilities in 7.20.0 are not present in
7.68.0 and 2 of them were still present on 2020-02-09, which also means 7.68.0
has 6 new vulnerabilities introduced in it but at some point after its release
maybe even after the date 2020-02-09. So, the number of vulnerabilities that
could have been avoided by using 7.68.0 on 2020-02-09 is 8 because they are
fixed in this version.

##### Our data set.

Overview of our collected data through these metrics is in the Figure 2.
First, second, and fourth subplots show the comparison of three metrics we
described earlier. Third subplot gives us the idea of how frequently updates
are released for software components. Looking at these plots all together
gives us an complete idea of the dynamic of update practice for these software
components, which individual plots don’t show. For e.g., if we just look at
the outdated by number of months, we could assume that the order of versions
of software components used from more outdated to less outdated would be:
cURL, Wget, OkHttp, and Requests. If we associate outdatedness with a
vulnerable state, the order of software components would be the same. If we
only look at the number of versions behind, the order of vulnerable states
would be cURL, OkHttp, Requests, and Wget. Although, this is not true because
Wget releases new versions less frequently than all of the other software
components. The more frequently a software component releases new versions; it
is more likely that versions behind of the used software components are going
to be high in the wild. Hence we can’t use only outdatedness by month or
versions. Number of avoidable vulnerabilities shows the true measure of how
vulnerable a software component is. Using that for our data, performance of
four software components from least secure to most secure is: cURL, Wget,
Python requests, and OkHTTP. Another thing to note is that in the first and
second sublot we could also see that some of the versions reported by IoT
devices for OkHttp are wrong as the number of days and versions behind is
negative because they were released in the future after they were used in the
wild. We looked further into this case and there is only one instance of false
reporting in our data.

##  4\. Findings

###  4.1. Outdated software components introduce avoidable vulnerabilities

Focusing on three factors of update practice, excluding the way vendors deploy
patches, our finding for all kinds of client hosts (including general purpose
computing and IoT devices) is consistent with previous studies. Nappa et al.
(Nappa et al., 2015) finds that devices move on to the latest version slowly
showing a long tail, and Sarabi et al. (Sarabi et al., 2017) models how long
they stay vulnerable (if they don”t update) using geometric distribution which
also shows long tail. Our findings using the three metrics in the graph Figure
3 and A.1 (Figure 6) for the four software over our capture duration suggest a
similar trend, which tells us that our data is valid and devices are not lying
about the user agents seen in our data.

In Figure 3, cURL is used as a user agent. On the horizontal axis we have a
number of versions behind for different versions of user agents seen at
different times during our data collection. The text label above the CDF for
the version is the mean number of months older version is from the latest
version available at the time of usage, for e.g., when a used version is 80
release versions behind, from the latest version it’s on average 140 months
(approximately 12 years) old. On the vertical axis on left, we have the
distribution of the number of vulnerabilities that could have been avoided for
every version behind, for e.g., when version behind is 25, number of avoidable
vulnerabilities are in the range 21-35 (median 28) for critical and high
severity vulnerabilities, and 24-39 (median 31.5) for vulnerabilities of all
severities. On the vertical axis on right, we have a CDF plotted for the
number of versions behind for 621 unique devices and cURL combinations. As per
our data for cURL, there is a 50% chance that when cURL is used as a user
agent it’s at least 14 release versions behind, 14 months old, and it could
have avoided a median of 11 vulnerabilities by using the latest available
version of that time. For cURL, at the long end of the tail we see 3 devices
using more than 90 release ($\sim$12 years) old versions.

Plot for all the versions shows a similar long tail of older release version
usage for all four software, see Figures 3, and A.1 (Figure 6). The number of
avoidable vulnerabilities is different and depends upon how many
vulnerabilities a software has. OkHtttp and Python Requests don’t have many
vulnerabilities, so updating to the latest version affects the vulnerability
state to a lesser degree. From the plot of cURL and Wget, we can say that
outdated versions over a period of time accrue more vulnerabilities and expand
the host’s vulnerability state. The previous statement is not true if a
software doesn’t have vulnerabilities, evident from the plot of Python
Requests, and OkHttp.

We see surprising downtrend in number of avoidable vulnerabilities after
number of version behind is more than 50 for cURL. cURL releases new version
every 49 (median) days. So 50 and 90 version behind cURL is approximately 7 to
12 years old. Presumably the reason for the downtrend could be that cURL had
less features 7 years ago, as its code base grew number of vulnerabilities
increased.

###  4.2. Update practices of IoT devices are different from general computing
devices

To find the distinction in update practice (including vendor deployment and
user installation) of IoT devices versus non-IoT devices we categorize devices
in six categories as per Fingerbank’s (”Fingerbank”, 2022) APIs. These six
categories are: ”IoT platforms”, ”IoT non-platforms” , ”Computing”, ”Storage
and Printers”, ”Networking”, and ”Others”, see Table 1.

Table 1. Device types in our 6 device categories based on Fingerbank API. Category |  Device types
---|---
IoT platforms |  smart TVs, set top boxes (STB), and digital video recorders (DVR)
IoT non-platforms |  rest of the IoT devices, such as smart cameras, smart assistants, speakers, smart vacuum cleaners
Computing |  computers, phones, tablets, and Raspberry Pi kind of devices
Storage and Printers |  network storage devices (NAS), and printer
Networking |  routers, wireless access points (WAP), switches, and firewalls
Other |  rest of the devices that can’t be categorized by Fingerbank

We distinguish between ”IoT platforms” and ”IoT non-platforms” as if an IoT
device allows third-party applications to be installed on them, if it does we
call them ”IoT Platform”. We distinguish ”IoT platforms” from ”IoT non-
platforms” because applications installed on them could be affecting the
version of user agents captured from those devices. ”Computing” category
includes general purpose computing devices. ”Storage and Printers” category
contains network storage devices (NAS) and printers. ”Networking” category
contains networking devices found in homes. Other categories include devices
that can’t be classified by Fingerbank API into the previous five categories.
Example of few vendors seen in our data are in Table 2.

Table 2. Examples of few vendors we have seen for four software in our data. Software component |  Few example vendors
---|---
cURL |  Samsung SmartThings, Google Nest, Parrot IoT, Vizio TV, WD TV Live, Meraki WAP, Synology NAS, Roomba, etc.
OkHttp |  Amazon Alexa and Fire TV, Google Chromecast and Home, Roomba, Vizio TV, NVidia Shield, Belkin Router, etc.
Wget |  WD TV Live, Netgear, Synology NAS, QNAP NAS, Amazon Alexa, Tablo DVR, etc.
Python Requests |  Eero WAP, Cisco WAP, Synology NAS, TP-Link, NVidia Shield, Telldus Smart home, etc.

We box plot the distribution of vulnerabilities avoided (affects the
vulnerability state of a device), number of versions behind, and number of
days behind for each user agent seen in the earlier mentioned six categories
for all the four software in Figures 4, and Appendix A.2. We would like to
note that for OkHttp, Figure 4 shows that older versions (60 months old) and
newer versions (20 - 25 months old) are equally vulnerable. This is because
OkHttp has only two vulnerabilities and they exist in versions from 2017, and
2019. So either used versions of OkHTTP are five years old (after 2014), or
two years old (after 2017) — they both contain two vulnerabilities and are
equally vulnerable.

Figure 4. Distribution of number of avoidable vulnerabilities, number of
versions behind, and number of months behind for all categories of IoT devices
using cURL (top), and OkHttp (bottom).

####  4.2.1. Comparing ”Computing” group with IoT devices

From Figures 4, and Appendix A.2 (Figure 7), we could see that each category
has a different distribution than others. By looking at the mean and median
number of avoidable vulnerabilities, we can clearly see that personal
computing devices have better software update posture than both the IoT device
categories for all the four software, except for cURL in ”IoT non-platforms”
category. We looked further into this case, and found that there are 105
unique Samsung SmartThings (Inc., 2022b) devices present in our dataset out of
163 devices in the ”IoT non-platforms” category. These SmartThings devices
consistently use the same cURL version 7.60.0, and are seen in a small burst
of time. At the time when they are seen they are only 6 versions behind from
the latest available version of cURL. These devices single-handedly improve
the update posture of ”IoT non-platforms” devices seen in our data by reducing
the mean and median number of avoidable vulnerabilities of ”IoT non-platforms”
category. Focusing on the update practice of SmartThings over our data
collection duration, we find their update practice is not good — they have two
kinds of devices and both of them use the same version of cURL for more than
20 months. We conclude that these devices might have released an update at the
time when we started data collection, and their presence in large numbers
skewed the results for ”IoT non-platforms category”. If we exclude Samsung
SmartThings devices, in the case of cURL, computing devices’ update practice
is better than ”IoT non-platforms” and ”IoT platforms” combined together. See
mean and median values of number of avoidable vulnerabilities with and without
SmartThings devices for these categories for cURL in Table 3.

Table 3. Number of avoidable vulnerabilities in ”Computing”, ”IoT platforms”, and ”IoT non-platforms” groups of cURL. Group |  mean |  median
---|---|---
Computing |  15.54 |  16.0
IoT platforms |  30.66 |  34.0
IoT non-platforms |  10.22 |  8.0
IoT non-platforms without Samsung SmartThings |  15.18 |  13.5
Both IoT groups together |  16.83 |  8.0
Both groups together without Samsung SmartThings |  24.76 |  28.0

####  4.2.2. Comparing ”IoT platforms” group with ”IoT non-platforms”

Focusing on IoT devices only, both the categories perform similarly in terms
of updates practice if we take a look at the mean and median values of
avoidable vulnerabilities for two IoT categories for all four software in the
Appendix B (Table 5), except for cURL for the reason explained in previous
subsection. ”IoT non-platforms” are doing better in general, which could be
probably because large amount of third-party applications on platforms don’t
have as consistent update practice as a single vendor IoT devices.

##### Takeaway:

as previous studies have only focused on general purpose computing devices, we
find that update practice of IoT devices are worse than computing devices.

###  4.3. Possible slow vendor deployment and end user installation in IoT
devices

In general purpose computing, prior work (Duebendorfer and Frei, 2009) has
suggested that silent updates lead to quicker update installation compared to
non-silent updates, and vendor update deployments depend upon vulnerability
disclosure (Nappa et al., 2015). For the software we used in our study,
vulnerability disclosure factor for update practices is the same for all
categories of devices, but the IoT group is using older versions for longer
duration. That means the remaining three factors — user promptness, vendor
deployment, and mechanism of deployment — are contributing to the reason why
update practice of IoT devices is worse than general purpose computing
devices. As these software are unlikely to be directly used by end users on
IoT devices, updates for them go through the IoT vendors. Regarding vendor
deployment factor in update, there has been an argument that vendors are bad
for security, because they add an additional pit stop for the patches getting
to end users (Laurie, 2008a, b). So maybe vendors are deploying patches slower
than the computing group. User promptness to install patches is affected by
update mechanism deployed by the vendors. Majority of well performing software
used on devices in computing groups have now adopted the silent update
strategy (Duebendorfer and Frei, 2009). Maybe IoT vendors are deploying
patches promptly but not using silent update mechanisms, presumably for
reasons like, not being able to do silent updates as IoT devices have
constraints on computational power. In contrast with computing groups, some
IoT devices are always on and don’t need user engagement after the setup. To
update IoT devices users have to use the companion apps, it could also be
possible that users set up the IoT device and forget to check the companion
app, which could be another reason for users’ not promptly installing updates.
Finding data relevant to all these factors could help us understand the actual
reason behind sluggishness in IoT group update practices.

Figure 5. For different vendors using cURL (top), and Wget (bottom),
distribution of versions used against time in left subplot; version seen in
incremental order with respect to time, along with number of avoidable
vulnerabilities and number of version behind in square brackets in right
subplot. Both the subplots show latest available version at the time of
version usage.

###  4.4. IoT vendors not deploying latest versions

To understand the vendor deployment factor of update practice we plotted
versions of four software used by user agent of different devices from
different vendors over the duration of our data capture. We only used devices
from ”IoT non-platforms” for these plots because ”IoT platforms” allow third
party apps. Versions seen from ”IoT platforms” category devices will not
depict a single vendor’s deployment practice. For both the vertical subplots
in the Figure 5, and A.3 (Figure 8) on the vertical axis we have different
versions of software used, and time on the horizontal axis. All software is
used by user agent of different vendors so we manually picked vendors which
were seen more frequently in our data. Different colors of distribution
represent different vendors as mentioned on the legends on the bottom right
corner of every figure. Latest available version with respect to time is
plotted in green color to give an understanding how far behind these used
versions are from latest version. Plots also mention the metrics number of
avoidable vulnerabilities and number of versions behind, described in section
3.3, in square brackets next to plotted distributions.

From these previous plots we can see the trend that user agents of different
devices of the same vendors are using the same version over a period of time.
We also see that some devices from a vendor have updated to a newer version
while some of them are still using older versions. This could possibly be
explained by either IoT vendors are deploying updates in a rolling fashion or
users are not installing the updates after the vendor deploy the updates. We
do see a jump in versions after a period, which is shown on the right side
subplot. We explain the right side plot in the next subsection 4.5.

###  4.5. Time to repair (TTR)

We also try to calculate time to repair per version in terms of days for each
software by calculating the number of days between two subsequent versions
seen in our data and divide it by the number of release versions differences
between them. A TTR is the slope between two points of the same vendor on the
right side subplot.

Right subplot plots only include points when time and version both were seen
in increasing order only. To elaborate, let’s suppose that in April 2019, we
see Alexa (Inc., 2022a) devices using version 2.4.0 of OkHttp on dates 1st,
10th, and 15. Again in 2019, we see older version 2.3.0 in May, a newer
version 3.5.0 in July on dates 1st and 15th, and another newer version 2.5 on
July 20th being used by Alexa devices. We include 2.4.0 used on 1st April and
3.5.0 used on 1st July in the right subplot because this is the first time we
have seen these versions. Clearly suggesting that the vendor has deployed a
patch as per our data. We don’t include 2.3.0, used in May because the device
has not been updated to a newer version, as we have seen newer versions from
Alexa devices before on 1st April. We don’t include 2.5.0 used on July 20th
even though it’s an increment in version from 2.4.0 because it might have been
rolled out by the vendor earlier, sometime between 1st April and 1st July,
it’s just not present in our data, as we see 3.5.0 being used before on July
1st.

Table 4. Calculated TTR per version in days, hours:minutes:seconds for few vendors. software |  vendor |  TTR per version
---|---|---
cURL |  Cisco Meraki |  1 days, 9:44:01
|  Cisco Meraki |  104 days, 0:30:41
|  Parrot |  0 days, 12:26:02
|  Parrot |  57 days, 20:46:07
|  Ruckus WAP |  2 days, 13:43:00
Python requests |  eero WAP |  0 days, 22:45:37
|  eero WAP |  320 days, 13:54:33
OkHttp |  Alexa |  0 days, 1:39:02
|  Alexa |  3 days, 16:50:06
|  Alexa |  6 days, 12:20:04
|  Alexa |  80 days, 9:25:09
|  TP-Link Tech |  63 days, 6:40:32

Using this methodology we attempt to calculate the TTR per version of each
vendor and report it in the Table 4. As our dataset is small, we have only few
points in the right subplot in Figures 5. So TTRs we have calculated are few
in terms of numbers. In the Table 4, the first small TTR for each version is
because vendors probably rolled out the updates shortly after we started the
capture, but the second ones are probably closer to actual TTR for the
respective vendor. From this data we can say TTR for different vendors for the
same software are different, consistent with (Nappa et al., 2015).

##### Takeaways from our findings:

  * •

IoT vendors are slower at deploying updates than all four software deploy
updates, which could leave end users vulnerable if software has patched
vulnerabilities in deployed updates.

  * •

Even when IoT vendors do deploy updates they don’t update to the latest
version.

  * •

In case of cURL, the updated version could have avoided vulnerabilities by
updating to the latest version available.

##  5\. Discussions and Future work

Our dataset is small but good enough for a preliminary study of update
practices in IoT ecosystem. Hopefully our preliminary findings are enough to
point out to the research community that more effort is needed to gain
understanding of update practice in the IoT ecosystem. We are planning to
collect more data in future so that we can get more consistent, statistically
large, and informative results, including (i) more device identification
information (as we currently rely on FingerBank’s API, which uses a blackbox
machine learning model with unclear accuracy); and (ii) more evidence of
software components beyond user agents (e.g., fingerprinting network traffic
with p0f to infer the operating system (p0f, 2022), or with TLS ClientHello to
infer the SSL library and versions (TLS, 2022)).

Three factors out of four that are collectively responsible for update
practice: user promptness, vendor update deployment, and mechanism of update
deployment, we couldn’t tell exactly which ones are contributing to worse
update practice in IoT devices, compared to general purpose computing devices.
Future studies could try to figure this out—for example, by asking IoT
Inspector users about their actual update behaviors, thus collecting human-
centered data alongside the User Agent data—so that efforts could be directed
in the right direction to improve IoT ecosystem update practice.

We recommend IoT device users to install updates promptly upon notification.
There are some IoT devices that don’t require user interaction after
installation, users should mindfully check for updates for these kinds of
devices and install them promptly when available. We recommend vendors to use
the latest versions of software components, even in cases when a component has
few vulnerabilities (for e.g. OkHttp) because with one critical vulnerability
attackers could do as much of damage as a software component having multiple
critical vulnerabilities. We would also recommend vendors to keep SBOMs and
track updates provided by software components they use. Learning from Linux
vendors (Laurie, 2008a, b; Yilek et al., 2009) , when software components
provide updates, vendors should deploy those updates as soon as possible
instead of acting as gatekeeper and blocking them because there is no other
way for IoT device users to update those components. Vendors should deploy
updates on IoT devices securely using standards like (Housley, 2005; sui,
2022), and silently (if possible) as previous studies have pointed out it’s
the most effective way (Gkantsidis et al., 2006; Frei et al., 2009;
Duebendorfer and Frei, 2009).

##  6\. Summary

We conduct a preliminary study of update practice of IoT devices and find that
it’s different from the findings of studies conducted about update practices
of general purpose computing devices. We find that IoT devices are slower to
update to newer versions compared to general purpose computing devices. We
couldn’t find the exact reason behind this but we narrowed down our findings
to where future research should direct their efforts in order to find out
which of three factors, end user promptness to install updates, vendor
deployment practice, and vendor deployment mechanism are responsible. We also
find that when IoT vendors deploy updates they don’t update to the latest
version, leaving hosts vulnerable to publicly known vulnerabilities.

## References

  * (1)
  * IoT (2021) 2021\.  The 1,200 IoT companies that are creating the connected world of the future – IoT Startup Landscape 2021.  https://iot-analytics.com/iot-startup-landscape/
  * UA- (2022) 2022\.  Browser detection using the user agent.  https://developer.mozilla.org/en-US/docs/Web/HTTP/Browser_detection_using_the_user_agent
  * TLS (2022) 2022\.  Fingerprinting OpenSSL libraries on IoT devices.  https://medium.com/all-things-inspected/fingerprinting-openssl-libraries-on-iot-devices-1d214b4d643
  * p0f (2022) 2022\.  p0f v3 (3.09b).  https://lcamtuf.coredump.cx/p0f3/
  * sui (2022) 2022\.  Software Updates for Internet of Things (suit).  https://datatracker.ietf.org/wg/suit/about/
  * Alhazmi and Malaiya (2005) O.H. Alhazmi and Y.K. Malaiya. 2005.  Modeling the vulnerability discovery process. In _16th IEEE International Symposium on Software Reliability Engineering (ISSRE’05)_. 10 pp.–138.  https://doi.org/10.1109/ISSRE.2005.30
  * Alhazmi et al. (2007) O.H. Alhazmi, Y.K. Malaiya, and I. Ray. 2007.  Measuring, analyzing and predicting security vulnerabilities in software systems.  _Computers & Security_ 26, 3 (2007), 219–228.  https://doi.org/10.1016/j.cose.2006.10.002
  * Antonakakis et al. (2017) Manos Antonakakis, Tim April, Michael Bailey, Matt Bernhard, Elie Bursztein, Jaime Cochran, Zakir Durumeric, J. Alex Halderman, Luca Invernizzi, Michalis Kallitsis, Deepak Kumar, Chaz Lever, Zane Ma, Joshua Mason, Damian Menscher, Chad Seaman, Nick Sullivan, Kurt Thomas, and Yi Zhou. 2017.  Understanding the Mirai Botnet. In _USENIX Security Symposium_.
  * Arbaugh et al. (2000) W.A. Arbaugh, W.L. Fithen, and J. McHugh. 2000.  Windows of vulnerability: a case study analysis.  _Computer_ 33, 12 (2000), 52–59.  https://doi.org/10.1109/2.889093
  * Arora et al. (2004) Ashish Arora, Ramayya Krishnan, Rahul Telang, and Yubao Yang. 2004.  Impact of Vulnerability Disclosure and Patch Availability - An Empirical Analysis. In _In Third Workshop on the Economics of Information Security_.
  * Bettayeb et al. (2019) Meriem Bettayeb, Qassim Nasir, and Manar Abu Talib. 2019\.  Firmware Update Attacks and Security for IoT Devices: Survey. In _Proceedings of the ArabWIC 6th Annual International Conference Research Track_ (Rabat, Morocco) _(ArabWIC 2019)_. Association for Computing Machinery, New York, NY, USA, Article 4, 6 pages.  https://doi.org/10.1145/3333165.3333169
  * Bilge et al. (2017) Leyla Bilge, Yufei Han, and Matteo Dell’Amico. 2017.  RiskTeller: Predicting the Risk of Cyber Incidents. In _Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security_ (Dallas, Texas, USA) _(CCS ’17)_. Association for Computing Machinery, New York, NY, USA, 1299–1311.  https://doi.org/10.1145/3133956.3134022
  * Cavusoglu et al. (2005) Hasan Cavusoglu, Huseyin Cavusoglu, and Srinivasan Raghunathan. 2005\.  Emerging Issues in Responsible Vulnerability Disclosure. In _WEIS_.
  * Clark et al. (2014) Sandy Clark, Michael Collis, Matt Blaze, and Jonathan M. Smith. 2014.  Moving Targets: Security and Rapid-Release in Firefox. In _Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security_ (Scottsdale, Arizona, USA) _(CCS ’14)_. Association for Computing Machinery, New York, NY, USA, 1256–1266.  https://doi.org/10.1145/2660267.2660320
  * ”cURL” (2022) ”cURL”. 2022.  _”Command line tool and library for transferring data with URLs”_.  "https://curl.se/"
  * DeKoven et al. (2019) Louis F. DeKoven, Audrey Randall, Ariana Mirian, Gautam Akiwate, Ansel Blume, Lawrence K. Saul, Aaron Schulman, Geoffrey M. Voelker, and Stefan Savage. 2019.  Measuring Security Practices and How They Impact Security. In _Proceedings of the Internet Measurement Conference_ (Amsterdam, Netherlands) _(IMC ’19)_. Association for Computing Machinery, New York, NY, USA, 36–49.  https://doi.org/10.1145/3355369.3355571
  * Duebendorfer and Frei (2009) Thomas Duebendorfer and Stefan Frei. 2009.  Web Browser Security Update Effectiveness. In _Proceedings of the 4th International Conference on Critical Information Infrastructures Security_ (Bonn, Germany) _(CRITIS’09)_. Springer-Verlag, Berlin, Heidelberg, 124–137.
  * Durumeric et al. (2014) Zakir Durumeric, Frank Li, James Kasten, Johanna Amann, Jethro Beekman, Mathias Payer, Nicolas Weaver, David Adrian, Vern Paxson, Michael Bailey, and J. Alex Halderman. 2014.  The Matter of Heartbleed. In _Proceedings of the 2014 Conference on Internet Measurement Conference_ (Vancouver, BC, Canada) _(IMC ’14)_. Association for Computing Machinery, New York, NY, USA, 475–488.  https://doi.org/10.1145/2663716.2663755
  * ”Fingerbank” (2022) ”Fingerbank”. 2022\.  _”ACCURATELY IDENTIFY CONNECTED DEVICES PERFORM ANOMALY DETECTION”_.  "https://www.fingerbank.org/about/"
  * Frei et al. (2009) Stefan Frei, Thomas Duebendorfer, and Bernhard Plattner. 2009\.  Firefox (In) Security Update Dynamics Exposed.  _SIGCOMM Comput. Commun. Rev._ 39, 1 (dec 2009), 16–22.  https://doi.org/10.1145/1496091.1496094
  * Gkantsidis et al. (2006) Christos Gkantsidis, Thomas Karagiannis, and Milan VojnoviC. 2006.  Planet Scale Software Updates.  _SIGCOMM Comput. Commun. Rev._ 36, 4 (aug 2006), 423–434.  https://doi.org/10.1145/1151659.1159961
  * Housley (2005) Russ Housley. 2005\.  Using Cryptographic Message Syntax (CMS) to Protect Firmware Packages.  RFC 4108.  https://doi.org/10.17487/RFC4108
  * Huang et al. (2020) Danny Yuxing Huang, Noah Apthorpe, Frank Li, Gunes Acar, and Nick Feamster. 2020.  IoT Inspector: Crowdsourcing Labeled Network Traffic from Smart Home Devices at Scale.  _Proc. ACM Interact. Mob. Wearable Ubiquitous Technol._ 4, 2, Article 46 (jun 2020), 21 pages.  https://doi.org/10.1145/3397333
  * Inc. (2022a) Amazon.com Inc. 2022a.  _Amazon Alexa_.  https://en.wikipedia.org/wiki/Amazon_Alexa
  * Inc. (2022b) SmartThings Inc. 2022b.  _Samsung SmartThings_.  https://en.wikipedia.org/wiki/SmartThings
  * Kandek (2009) Wolfgang Kandek. 2009\.  _The Laws of Vulnerabilities 2.0._ https://www.qualys.com/docs/laws-of-vulnerabilities-2.0.pdf
  * Khan et al. (2012) Moazzam Khan, Zehui Bi, and John A. Copeland. 2012.  Software updates as a security metric: Passive identification of update trends and effect on machine infection. In _MILCOM 2012 - 2012 IEEE Military Communications Conference_. 1–6.  https://doi.org/10.1109/MILCOM.2012.6415869
  * Kim et al. (2018) Dae-Young Kim, Seokhoon Kim, and Jong Hyuk Park. 2018\.  Remote Software Update in Trusted Connection of Long Range IoT Networking Integrated With Mobile Edge Cloud.  _IEEE Access_ 6 (2018), 66831–66840.  https://doi.org/10.1109/ACCESS.2017.2774239
  * Kumar et al. (2019) Deepak Kumar, Kelly Shen, Benton Case, Deepali Garg, Galina Alperovich, Dmitry Kuznetsov, Rajarshi Gupta, and Zakir Durumeric. 2019\.  All Things Considered: An Analysis of $\\{$IoT$\\}$ Devices on Home Networks. In _28th USENIX security symposium (USENIX Security 19)_. 1169–1185.
  * Langiu et al. (2019) Antonio Langiu, Carlo Alberto Boano, Markus Schuß, and Kay Römer. 2019.  UpKit: An Open-Source, Portable, and Lightweight Update Framework for Constrained IoT Devices. In _2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS)_. 2101–2112.  https://doi.org/10.1109/ICDCS.2019.00207
  * Laurie (2008a) Ben Laurie. 2008a.  _Debian and OpenSSL: The Aftermath_.  https://www.links.org/?p=328
  * Laurie (2008b) Ben Laurie. 2008b.  _Vendors Are Bad For Security_.  https://www.links.org/?p=327
  * Mohajeri Moghaddam et al. (2019) Hooman Mohajeri Moghaddam, Gunes Acar, Ben Burgess, Arunesh Mathur, Danny Yuxing Huang, Nick Feamster, Edward W Felten, Prateek Mittal, and Arvind Narayanan. 2019\.  Watching you watch: The tracking ecosystem of over-the-top tv streaming devices. In _Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security_. 131–147.
  * Morgner et al. (2020) Philipp Morgner, Christoph Mai, Nicole Koschate-Fischer, Felix Freiling, and Zinaida Benenson. 2020\.  Security Update Labels: Establishing Economic Incentives for Security Patching of IoT Consumer Products. In _2020 IEEE Symposium on Security and Privacy (SP)_. 429–446.  https://doi.org/10.1109/SP40000.2020.00021
  * Nappa et al. (2015) Antonio Nappa, Richard Johnson, Leyla Bilge, Juan Caballero, and Tudor Dumitras. 2015.  The Attack of the Clones: A Study of the Impact of Shared Code on Vulnerability Patching. In _2015 IEEE Symposium on Security and Privacy_. 692–708.  https://doi.org/10.1109/SP.2015.48
  * Neuhaus et al. (2007) Stephan Neuhaus, Thomas Zimmermann, Christian Holler, and Andreas Zeller. 2007.  Predicting Vulnerable Software Components. In _Proceedings of the 14th ACM Conference on Computer and Communications Security_ (Alexandria, Virginia, USA) _(CCS ’07)_. Association for Computing Machinery, New York, NY, USA, 529–540.  https://doi.org/10.1145/1315245.1315311
  * NVD (2022) NVD. 2022.  _National Vulnerability Database_.  NIST.  https://nvd.nist.gov/
  * ”OkHttp” (2022) ”OkHttp”. 2022\.  _”OkHttp HTTP client”_.  "https://square.github.io/okhttp/"
  * Paracha et al. (2021) Muhammad Talha Paracha, Daniel J Dubois, Narseo Vallina-Rodriguez, and David Choffnes. 2021\.  IoTLS: understanding TLS usage in consumer IoT devices. In _Proceedings of the 21st ACM Internet Measurement Conference_. 165–178.
  * Ramos (2006) Terry Ramos. 2006\.  _The Laws of Vulnerabilities_.  https://www.qualys.com/docs/laws-of-vulnerabilities-presentation.pdf
  * Reeder et al. (2017) Robert W. Reeder, Iulia Ion, and Sunny Consolvo. 2017\.  152 Simple Steps to Stay Safe Online: Security Advice for Non-Tech-Savvy Users.  _IEEE Security and Privacy_ (2017).
  * Ren et al. (2019) Jingjing Ren, Daniel J. Dubois, David R. Choffnes, Anna Maria Mandalari, Roman Kolcun, and Hamed Haddadi. 2019.  Information Exposure From Consumer IoT Devices: A Multidimensional, Network-Informed Measurement Approach. In _Proceedings of the Internet Measurement Conference_. 267–279.
  * Requests” (2022) ”Python Requests”. 2022\.  _”an elegant and simple HTTP library for Python, built for human beings”_.  "https://en.wikipedia.org/wiki/Requests_(software)"
  * Rescorla (2003) Eric Rescorla. 2003\.  Security Holes . . . Who Cares?. In _12th USENIX Security Symposium (USENIX Security 03)_. USENIX Association, Washington, D.C.  https://www.usenix.org/conference/12th-usenix-security-symposium/security-holes-who-cares
  * Rescorla (2005) Eric Rescorla. 2005\.  Is Finding Security Holes a Good Idea?  _IEEE Security and Privacy_ 3, 1 (jan 2005), 14–19.  https://doi.org/10.1109/MSP.2005.17
  * Sarabi et al. (2017) Armin Sarabi, Ziyun Zhu, Chaowei Xiao, Mingyan Liu, and Tudor Dumitras. 2017.  Patch Me If You Can: A Study on the Effects of Individual User Behavior on the End-Host Vulnerability State. In _Proceedings of the 18th Passive and Active Measurement PAM._ ACM, Sydney, Australia.
  * Shahzad et al. (2012) Muhammad Shahzad, Muhammad Zubair Shafiq, and Alex X. Liu. 2012.  A large scale exploratory analysis of software vulnerability life cycles. In _2012 34th International Conference on Software Engineering (ICSE)_. 771–781.  https://doi.org/10.1109/ICSE.2012.6227141
  * ”Wget” (2022) ”Wget”. 2022.  _”Retrieve files via HTTP or FTP”_.  "https://en.wikipedia.org/wiki/Wget"
  * Yilek et al. (2009) Scott Yilek, Eric Rescorla, Hovav Shacham, Brandon Enright, and Stefan Savage. 2009\.  When Private Keys Are Public: Results from the 2008 Debian OpenSSL Vulnerability. In _Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement_ (Chicago, Illinois, USA) _(IMC ’09)_. Association for Computing Machinery, New York, NY, USA, 15–27.  https://doi.org/10.1145/1644893.1644896
  * Yohan and Lo (2018) Alexander Yohan and Nai-Wei Lo. 2018.  An Over-the-Blockchain Firmware Update Framework for IoT Devices. In _2018 IEEE Conference on Dependable and Secure Computing (DSC)_. 1–8.  https://doi.org/10.1109/DESEC.2018.8625164
  * Yousefnezhad et al. (2020) Narges Yousefnezhad, Avleen Malhi, and Kary Främling. 2020\.  Security in product lifecycle of IoT devices: A survey.  _Journal of Network and Computer Applications_ 171 (2020), 102779\.  https://doi.org/10.1016/j.jnca.2020.102779
  * Zhang et al. (2016) Chi Zhang, Wonsun Ahn, Youtao Zhang, and Bruce R. Childers. 2016\.  Live code update for IoT devices in energy harvesting environments. In _2016 5th Non-Volatile Memory Systems and Applications Symposium (NVMSA)_. 1–6.  https://doi.org/10.1109/NVMSA.2016.7547182

Figure 6. Number of avoidable vulnerabilities in Wget, OkHttp, and Python
Requests, from top to bottom, against number of versions behind, and CDF
distribution of devices for number of versions behind.

Figure 7. Distribution of number of avoidable vulnerabilities, number of versions behind, and number of months behind for all categories of IoT devices using Wget (top), and Python Requests (bottom). Figure 8. For different vendors using Python Requests, distribution of versions used against time in left subplot; version seen in incremental order with respect to time, along with number of avoidable vulnerabilities and number of version behind in square brackets in right subplot. Both the subplots show latest available version at the time of version usage. Table 5. Table of mean, and median (in parenthesis) for each categories of IoT devices’ distribution of number of avoidable vulnerabilities, number of versions behind, and number of months behind for all software. prog |  metric |  IoT platforms |  IoT non-platforms |  Storage & Printers |  Networking |  Computing |  Others
---|---|---|---|---|---|---|---
cURL |  avoidable hi+ vulnerabilities |  30.67 (34.00) |  10.22 (8.00) |  21.83 (16.00) |  21.05 (18.00) |  15.54 (16.00) |  14.31 (8.00)
|  versions behind |  40.71 (30.00) |  10.58 (6.00) |  21.67 (15.00) |  36.60 (21.00) |  17.23 (15.00) |  16.60 (6.00)
|  months behind |  61.35 (40.10) |  17.09 (10.50) |  31.50 (23.57) |  57.34 (31.20) |  25.50 (23.57) |  25.45 (10.50)
Wget |  avoidable hi+ vulnerabilities |  7.00 (7.00) |  4.60 (7.00) |  5.18 (6.00) |  7.00 (7.00) |  4.44 (4.00) |  4.55 (6.00)
|  versions behind |  18.40 (17.00) |  12.40 (18.00) |  12.53 (12.00) |  25.00 (25.50) |  11.47 (10.00) |  13.00 (16.00)
|  months behind |  73.93 (63.40) |  57.75 (79.70) |  44.03 (41.23) |  119.69 (123.55) |  40.64 (34.33) |  55.47 (54.03)
OkHttp |  avoidable vulnerabilities |  0.82 (1.00) |  0.65 (1.00) |  NaN |  0.38 (0.00) |  0.63 (1.00) |  0.75 (1.00)
|  versions behind |  25.09 (25.00) |  27.38 (28.00) |  NaN |  22.50 (17.50) |  19.82 (19.00) |  18.32 (16.00)
|  months behind |  32.87 (38.60) |  34.24 (32.67) |  NaN |  15.11 (9.97) |  20.04 (17.25) |  23.26 (19.47)
Requests |  avoidable vulnerabilities |  0.50 (0.50) |  0.75 (1.00) |  0.67 (1.00) |  1.95 (2.00) |  0.18 (0.00) |  0.22 (0.00)
|  versions behind |  16.00 (16.00) |  21.42 (27.50) |  15.33 (5.00) |  45.13 (46.00) |  4.58 (0.00) |  6.72 (1.00)
|  months behind |  24.17 (24.17) |  30.63 (37.47) |  21.72 (16.07) |  48.15 (48.27) |  7.61 (0.00) |  9.76 (1.42)

##  Appendix A Plots

###  A.1. Avoidable vulnerabilities vs versions behind for all kinds of
devices

Figures in the Appendix A.1 shows the plots for avoidable vulnerabilities vs
versions behind for Wget, OkHttp, and Python Requests in Figure 6. The number
of avoidable vulnerabilities and CDF of Wget, OkHttp, and Python request shows
similar trend to Curl, see Figure 3. The shape of OkHttp and Requests appear
to be different because they have fewer vulnerabilities, and that results in
zoomed in Y axis; possibly because OkHttp and Requests are written in memory
safe languages Java, and Python respectively. Otherwise, OkHttp and Requests
also show a long tail distribution. To list the software components in order
form most vulnerable to secure, we could look at number of avoidable
vulnerabilities in those components, and from that Curl seems most vulnerable,
then Wget, Python Requests, and finally OkHttp.

By looking at the plots of Python Requests and OkHttp, one could make an
argument that if software component has less vulnerabilities, using newer
versions doesn’t result in improved the security, but those few
vulnerabilities in older version could be exploited by attackers and do as
much damage as having large number of vulnerabilities. Unless a software has
no vulnerabilities, it’s always beneficial to use the latest version.

####  A.1.1. Plot of Wget, OkHttp, and Python Requests is in Figure 6

###  A.2. Distribution of number of avoidable vulnerabilities, number of
versions behind, and number of month behind

Figures in the A.2 shows the distribution of distribution of number of
avoidable vulnerabilities, number of versions behind, and number of month
behind for different categories for Wget, and Python Requests in Figure 7.
From both the plots we can see that distribution for different categories are
different. Looking at the mean and median, update practices of computing group
is better than both IoT non-platforms and IoT platforms.

####  A.2.1. Plot of Wget and Python Requests in Figure 7

###  A.3. Plots for versions of software components used by vendors over our
capture duration

Figures in the appendix A.3 depicts the version of Python Requests used by
vendor eero Wap. Similar to Figure 5 for cURL and OkHttp, we see that vendor
has rolled out an updates to versions 2.21.0 and 2.23.0 and these updates are
1 and 3 versions behind from latest available at that time respectively.

We have not included the plots for Wget because we did not see any vendor
updating Wget in our data.

####  A.3.1. Plot of Python Requests in Figure 8

##  Appendix B Statistical values of metrics for all software components

Table 5 lists the mean and median values of metrics used by us.