# Humans-as-a-sensor for buildings: Intensive longitudinal indoor comfort
models

Prageeth Jayathissa, Matias Quintana, Mahmoud Abdelrahman, and Clayton Miller
∗ Building and Urban Data Science (BUDS) Lab, National University of Singapore
(NUS), Singapore  ∗Corresponding Author<EMAIL_ADDRESS>+65 81602452

###### Abstract

Evaluating and optimising human comfort within the built environment is
challenging due to the large number of physiological, psychological and
environmental variables that affect occupant comfort preference. Human
perception could be helpful to capture these disparate phenomena and
interpreting their impact; the challenge is collecting spatially and
temporally diverse subjective feedback in a scalable way. This paper presents
a methodology to collect intensive longitudinal subjective feedback of
comfort-based preference using micro ecological momentary assessments on a
smartwatch platform. An experiment with 30 occupants over two weeks produced
4,378 field-based surveys for thermal, noise, and acoustic preference. The
occupants and the spaces in which they left feedback were then clustered
according to these preference tendencies. These groups were used to create
different feature sets with combinations of environmental and physiological
variables, for use in a multi-class classification task. These classification
models were trained on a feature set that was developed from time-series
attributes, environmental and near-body sensors, heart rate, and the
historical preferences of both the individual and the comfort group assigned.
The most accurate model had multi-class classification F1 micro scores of 64%,
80% and 86% for thermal, light, and noise preference, respectively. The
discussion outlines how these models can enhance comfort preference prediction
when supplementing data from installed sensors. The approach presented prompts
reflection on how the building analysis community evaluates, controls, and
designs indoor environments through balancing the measurement of variables
with _strategically asking for occupant preferences_ in an intensive
longitudinal way.

###### keywords:

Indoor environmental quality , Thermal comfort models , Personalised comfort
model , Machine learning , Ecological momentary assessment , Occupant-centric
, Occupant behaviour

††journal: Buildings

##  1 Introduction

Many office workers are familiar with the battle of the thermostat, or that
co-worker who talks loudly on the phone. Many researchers in indoor comfort
are also aware of the high rates of discomfort amongst office workers [1, 2].
Vast global efforts have been undertaken to evaluate this discomfort, and with
that knowledge, build models that can be used for the design and control of
buildings. In the realm of thermal comfort, for example, two dominant models
are in use. The first is the _Predicted Mean Vote (PMV)_ that models comfort
based on heat transfer characteristics between the human and their surrounding
environment [3]. The other, more modern version, is the _Adaptive Comfort_
model that includes the human adaptability to climate, drawing a linear
relationship between the indoor and outdoor environments [4].

The underlying issue with modelling human comfort is the sheer number of
variables present and the difficulty in accurately measuring them. Figure 1
highlights this issue by detailing a list of studied physiological,
psychological, and environmental variables that influence thermal, visual, and
aural comfort. While the empirical models in the academic literature are
capable of incorporating a handful of these variables, the exclusion of the
rest can cause significant errors. One reason is that the interrelationship
between different indoor environmental parameters is not well-known [5]. _It
was shown in a recent study that the lowest indoor environmental satisfaction
factor drives the overall satisfaction[6]_. For example, while one can measure
the temperature and humidity of a room; the type of meal a person ate, and
even the spices present in the meal, can put the human body in a different
state of thermal perception [7, 8]. Furthermore, most of the studies that
depend on measuring environmental variables using mobile carts with mounted
sensors [9, 10, 11] or low-cost continuous sensing sensors [12] face problems
related to the accuracy and calibration [13]. While being comprehensive in
capturing most comfort-related factors, Figure 1 excludes literature about
physical and mental ailments, which further adds variance to the models.

Figure 1: Graphical review of physiological, psychological and environmental
factors influencing human comfort. Thermal - clockwise from top left:
adaptation to outdoor environment [4], air flow [14], solar radiation [15],
circadian rhythm [16], daylight perception [17], environmental long wave
radiation [18], perspiration [19], diet-induced thermo-genesis [8],
subcutaneous fat thickness [20], posture [21], temperature/humidity [3] [22],
physical activity [23], clothing level [24], vascular anatomy [25], near body
heat sources [26], gender [27], age [28], basal metabolic rate [29], thermal
regulation [30] [31], alliesthesia [32]. Visual - clockwise from top left:
circadian calibration [33], daylight [34], view [35], spectrum [36],
uniformity [37], illuminance [38], flicker [39], susceptibility to migraines
[40], biophilia [41], glare [42]. Aural - clockwise from top: seniority in a
company [43], subjective sensitivity [44], sound privacy [45], sound
absorption [46], controllability [47], task [48], variable and constant noise
[49].

It is, therefore, not surprising to find that preference prediction models
with only a handful of the factors in Figure 1 have low accuracy. For example,
the previously mentioned PMV model uses personal and environmental parameters
such as temperature, humidity, mean radiant temperature, air movement, and
clothing and metabolism levels to predict thermal comfort. A recent analysis
showed that this model is _only accurate 34% of the time_ [50]. In the control
of real buildings, these models are further simplified, and it is usually the
only variables of temperature, illuminance, and noise levels that are used to
evaluate thermal, visual, and aural comfort, respectively.

###  1.1 Can longitudinal human perception feedback supplement sensors?

The human nervous system detects sensation and converts it into thoughts and
feelings, which are the very foundation of the word _comfort_. What if
occupants in buildings were asked about their subjective preference in spaces,
instead of only measuring environmental variables and using them to infer
comfort? Collecting enough comfort preference feedback from a single person
over days or weeks would take advantage of a human’s ability to evaluate
dozens of variables simultaneously, including those that are difficult to
measure. How can this type of methodology be accomplished in a scalable way
without annoying occupants too much or inducing survey fatigue? Can this
approach provide insight into comfort problem areas that contemporary sensors
are too expensive or problematic in implementation?

The goal of this paper was to test the ability of an intensive longitudinal
method to capture numerous environmental feedback data from experimental
participants in a field setting. This study uses _Micro Ecological Momentary
Assessments (EMA)_ as a subjective feedback methodology that overcomes many of
the challenges presented by traditional methods [51]. Micro-EMA is a method of
using a smartwatch interface to prompt and collect momentary, _right-here-
right-now_ subjective feedback from a single person over several weeks [52].
Receiving a large amount of feedback from a single person in a diversity of
spaces and comfort exposures provided the ability to understand the comfort
_preference tendencies_ of a person. It is proposed that these behavioural
tendencies can be used to segment people into groups related to how they
perceive their environment. Grouping people with similar comfort preferences
could, therefore, increase the accuracy of predicting where a person will be
comfortable and what the system can to respond without additional sensors.
Additionally, collecting large amounts of subjective preference data from
numerous people in a particular space can characterise the comfort-related
attributes of that space _to supplement data being collected from the sensors
installed_. If technically scalable and not too disruptive to an occupant,
using _humans-as-a-sensor_ in buildings could change the way post-occupancy
evaluations, building and system design, and controls and automation are done.
There would be opportunities for people to provide feedback for short-term
episodic uses (days or weeks), for building commissioning or long-term (months
or years), and for continuous system control and management. This work
complements the momentum from other disciplines focused on the use of humans
as sensors for applications in detection of events using social media data
[53], for detecting emergencies [54], and for cybersecurity [55].

Figure 2:  The cozie watch-face, built on the Fitbit smartwatch platform was
used to collect subjective feedback. The phone that is paired with the Fitbit
can be used to set up additional questions.

###  1.2 Paper overview

This paper presents how high-frequency micro-EMA, combined with sensor data
time-series analysis, can enable the evaluation, control, and rethinking of
the design of indoor environments. Section 2 first gives a more detailed
overview of foundational work in indoor preference capture and modelling and
the novelty being proposed. Section 3 provides a comprehensive explanation of
the design and deployment of a smart watch-based subjective preference data
collection and environmental variable measurement system. Section 4 details
the results from a field-based implementation at [removed for anonymization]
and the testing of various preference models based on intensive longitudinal
data. Finally, Section 5 and 6 discusses integration methods in buildings,
limitations, future work, and details on how to reproduce the study using open
data and code.

##  2 Background and novelty

This work builds upon previous literature focused on the measurement of
factors that may influence thermal, visual, and aural comfort in the built
environment. These modelling techniques are converged with an intensive
longitudinal experience sampling technique that is common in the medical and
psychological communities, but only emerging in the analysis of buildings.
This section covers previous work in the building context using intensive
longitudinal data and an overview of the novelty of the work in this paper as
compared to the literature.

###  2.1 Indoor environmental comfort variables and models

There are generally two models types used in the literature for indoor comfort
assessment: 1) objective-subjective, and 2) objective-criteria [56]. Which
method to use is decided based on the aim of the evaluation. On the one hand,
the _objective-subjective_ model combines the indoor environmental
measurements from sensors with the subjective feedback from users, mostly in
the form of post-occupancy evaluation (POE) surveys [57, 58, 59, 60]. On the
other hand, the _objective-criteria_ model is used in ranking or rating a
building by comparing the indoor measurements from IEQ sensors with building
performance measurement protocols such as LEED or WELL certifications [11].
Both of the methods have drawbacks both in measuring the environmental data as
well as surveying occupants [56].

In terms of environmental measurements, work has been done that used accurate
sensors that were mounted on movable carts [61, 11]. However, these sensors
were not affordable to all building operations scenarios [56]. The
affordability challenge was met using low-cost continuous sensing sensors that
required frequent calibration [12]. Nevertheless, the location of these
sensors in buildings, and interpolation of the readings still represented a
challenge in the literature, given the fact that indoor spaces are
heterogeneous [62]. On the other side of the spectrum, surveys pose some
problems related to questions, e.g. what to ask, whom to ask, and how to
interpret the results [56]. Additionally, Porter et al. [63] discussed the
term _survey fatigue_ in which users feel overwhelmed by questions that may
lead to a misrepresentation in responses and reduced response rates.

A related area of recent focus is the use of wearable and infrared radiation
sensors to capture _near-body_ physiological data that define the
environmental conditions close to or at the skin surface of an occupant. A
recent study focused on creating personalised comfort models from these data
in the context of field-based deployment on 14 subjects [64]. This deployment
and the models produced used wrist and ankle skin temperature from several
sensors placed on the participants and a smartphone application to collect
surveys. Further work in the indoor context showed that both wearable sensors
and infrared radiation cameras led to a 3-4% increase in accuracy of thermal
comfort sensation prediction, marginally justifying the cost of implementation
in a field setting [65].

###  2.2 Ecological momentary assessments (EMA)

The next area of background focuses on the challenge of collecting large
amounts of longitudinal data from a person. Many fields of study have relied
upon the ecological momentary assessment [51] methodology to meet this
challenge. This method is a type of intensive longitudinal experience sampling
most often utilised in studying human behaviour. The word _ecological_
describes that fact that the measurement is taken in the subjects’ natural
environment without impacting their task at hand. The word _momentary_
pertains to the fact that feedback is requested at the moment of experience,
as opposed to asking a subject to recall a past experience. And finally, the
_assessments_ are not static one-off outcomes but occur over time, thus
accounting for temporal dynamics. Traditional models found in literature such
as surveys are insufficient as their sampling rates are low, require the
occupant to completely stop their task at hand to focus on the survey, and in
many cases, ask for a recollection of past experiences. There is the further
issue of survey fatigue [63] and even when willing to participate, there is a
concern about how accurate their responses are [66]. The use of a smartwatch
for data collection, coined _micro ecological momentary assessments_ , has
been shown to be so user-friendly that it does not significantly disrupt any
ongoing activity [52]. Furthermore, an eight-fold increase in sampling
frequency can be obtained, in comparison to smartphone use, without burdening
the user. Recent work has used ecological momentary assessments to assess the
built environment through the use of smartphones [67]. While such applications
are a step in the right direction, they were only able to collect eight
feedback points per occupant, which is insufficient for time-series analysis.

###  2.3 Similar work in intensive longitudinal data collection in the built
environment

Intensive longitudinal methodologies have begun to emerge as a way to
characterise occupants for various built environment objectives. In the urban
context, several studies have deployed sensors on people to understand their
experiences across their daily lives. A large study based in Singapore used
thousands of wearable sensors in populations of students to discover travel
patterns [68], collect information about thermal parameters [69], and even
infer the impact of public spaces on happiness [70]. Work has been done in a
controlled outdoor field study to understand the impact of the urban context
on various emotions and physiological responses of human [71]. In the indoor
setting, targeted work on collecting longitudinal data for more specific
purposes has also emerged. The previously mentioned wearable study focusing on
thermal comfort collected numerous data from the 14 participants over the 2-4
week study [64]. Another recent study that deployed a cyber-physical system to
collect longitudinal data in offices focused on occupant concentration [72].
The work in this paper is most directly related to previous work in collecting
longitudinal comfort feedback from smartphone interfaces for the allocation of
activity-based workspaces [73] and through a sustainability tour in a
university campus building [74].

###  2.4 Novelty of proposed approach

Despite the momentum in field-based intensive longitudinal methodologies,
there are still several barriers to their implementation in real-world
settings. Not the least is the challenge of getting human occupants to give
data for comfort surveys, install applications, or wear devices. Working from
this knowledge, the authors developed _cozie_ , as seen in Figure 2, an open-
source, smartwatch clock-face designed to conduct micro-EMA surveys for high-
frequency data collection [75]. The application is open-sourced and free to
download and use on the Fitbit gallery 111https://cozie.app/.

Figure 3: Overview of the experimental deployment in the SDE buildings in four
distinct tiers: Tier 1 is the base methodology which is production-ready for
real-building deployment. It requires a smart watch with the cozie clock-face
installed. Tier 1b, is an extension to the base methodology by adding a
temperature sensor to the watch. Tier 2 includes building-wide indoor
localisation. In this experiment, Steerpath Bluetooth beacons were used, which
communicate with the occupant’s smartphone to determine the occupant’s
location. Tier 3 merges the localised feedback points with environmental
sensors in the same comfort zone as the occupant.

The innovations outlined in this work as compared to the previously mentioned
studies are:

  * 1.

The hardware and software deployment methodology have a focus on practicality
in scalable, field-based implementations. Experimental participants were only
asked to wear a single smartwatch device and answer survey questions that
utilise a relatively small amount of time. The focus was on testing a
configuration that was easily applied in a real-world context. The modelling
methodology was designed to capture as much signal as possible in the field
setting without the ability to control and verify sensor proximity and
accuracy consistently.

  * 2.

A series of pre-processing steps were developed to convert intensive
longitudinal data into model input features that characterise the _tendency of
groups of people to have similar comfort preferences_ , sometimes independent
of the objective environmental factors such as temperature. A simple example
of this concept is the commonly discussed, yet often anecdotal, person who
seems always to need more cooling, even when the temperature is already low
relative to the comfort zone. In this study, clustering was used to group
people into _comfort preference types_ as an input feature to a preference
prediction model.

  * 3.

This paper introduces and tests a simple form of a _cold start_ variant to the
preference models that could be used to predict an occupant’s preference with
limited or no data about their preference history in a particular space or
according to particular objective measurements such as temperature, humidity,
or other factors. This model enables the deployment of the _cozie_ data
collection methodology by a set of participants in a building and then the
creation of prediction models that could accommodate future occupants
regardless of whether they have worn a smartwatch in those spaces.

  * 4.

The process seeks to show that comfort-based preference prediction can be
accurate even in the absence of environmental sensors if enough intensive
longitudinal data has been collected from enough occupants. The context of
this experiment was in relatively uncontrolled, field-based settings as
opposed to laboratory conditions.

##  3 Methodology

To collect intensive longitudinal data in a field setting, the _cozie_
platform was built on the Fitbit smart watch222https://www.fitbit.com/ and
various time-series database technologies. In this section, the details of
this technology stack are explained in the context of a deployment on 30 test
participants in buildings at the National University of Singapore (NUS) in the
School of Design and Environment (SDE). The definition of an _occupant_ in
this study was a test participant who wore the smartwatch, and a _manager_ as
the person who coordinated the study. Thirty participants were recruited via
an online form, were compared to the inclusion criteria for the study, and
were on-boarded according to an approved ethics review application. Priority
was given to participants who work full time in the SDE-related buildings on
campus, and they were selected to maintain an even gender distribution.

The technology used in the deployment of this study can be sub-sectioned into
individual tiers as described in Figure 3, with each level requiring
additional resources to implement. For the experiment in SDE4, all tiers were
incorporated.

###  3.1 Tier 1: Smartwatch for micro-EMA

Tier 1 is the core methodology presented in this paper, which uses the _cozie_
clock-face, as shown in Figure 2. The occupants were asked to wear a Fitbit
Versa smartwatch during daytime hours while on the NUS campus at the very
least but were also welcome to wear the device for the entire duration of the
study. Participants were asked to leave momentary assessment feedback on their
comfort preferences at different points throughout the day on the watch face
of the Fitbit device. Each time they responded to the survey, they were asked
about their thermal, visual and aural preference using the options found in
Figure 2. Comfort preference was chosen as the feedback most applicable to the
methodology due to a three-point scale that is most appropriate for frequent
watch-based surveys. Preference surveys also provide more meaningful
information by indicating how the occupant would want the environment to
change as opposed to _satisfaction_ or _sensation_ survey types that only
capture how the occupant feels. The participants were asked to answer the
questions when they moved from one environment to another, which amounted to
approximately 5-15 assessments per day. The smartwatch also prompted the
occupants with a small vibration that requested feedback from them at
different timed points in the day. This prompt only occurred during daytime
hours when the subject was active. The momentary assessment took less than 15
seconds to complete. Throughout the experiment, the cumulative amount of time
spent answering the momentary assessment was approximately 20-40 minutes.

Detailed documentation for using _cozie_ , along with the source code to an
open-source repository, can be found on a GitHub
repository333https://github.com/buds-lab/cozie. The platform also can collect
sensation, satisfaction and objective feedback such as clothing and activity
levels. These features were added after the experiments outlined in this paper
and were not used in this study.

###  3.2 Tier 2: Indoor localisation

Tier 1 is likely sufficient for experiments conducted in small office spaces.
If only a few different zones exist, then an occupant’s location could be
quickly determined through a supplementary question in the question flow of
the survey. However, in a large building, such as the SDE4 building where the
outlined experiment was conducted, a more sophisticated indoor localisation
system was required. The SDE4 building has six different floors, a gross floor
area of around 8,500 square meters, and a large variety of different indoor
environments. To determine an occupant’s location in a building, 100 Bluetooth
beacons and the Steerpath444https://steerpath.com/ platform were installed
throughout the building. These beacons communicated with a custom-built
smartphone application, called the _Yak App_ [76], to determine their location
with a one-meter precision. The location data was then used to geo-fence the
occupant within various zones of the building and was merged with the
subjective preference feedback data in the cloud.

###  3.3 Tier 3: Preference data convergence with environmental sensors

Tier 3 included the deployment of 45 indoor and outdoor environmental quality
(IEQ) sensors in the experimental context. This data collection tier was used
to compare the results of the subjective feedback, with existing environmental
models. The IEQ sensors were WiFi-connected and were deployed by the company
SenSING555https://sensing.online/ as part of an installation of sensors
campus-wide. These sensor kits measured temperature, humidity, noise level,
and illuminance. At least one sensor device was installed in each zone of the
building, and the data was pulled from an API and merged with the subjective
preference data in the cloud.

###  3.4 Tier 1b: Strap-mounted sensor kit

Tier 3b included a temperature sensor was used from mbient
labs666https://mbientlab.com/, which was attached to the watch through a
custom three dimensional (3D) printed case. The design file for this case can
be found
online777https://myhub.autodesk360.com/ue29ab3ac/g/shares/SH919a0QTf3c32634dcfe0a71457c4729699.
The mbient device logged data locally, which was transferred to the cloud
database at the end of the experiment.

###  3.5 Occupant and room preference clustering

This analysis included the hypothesis that the feedback of one of the
occupants in such groups could be used to characterise the preferences of all
group members for a particular space or set of conditions. In this step, the
preference history of occupants was used to do a simple clustering-based
segmentation step to group occupants according to their raw feedback
preference tendencies. For example, occupants who more frequently indicated
_prefer cooler_ as compared to a _no change_ would be grouped. This strategy
was a simplified version of this type of clustering as it neglects other
context-based variables (environmental and physiological measurements). This
choice was made to keep the method feasible even in situations in which other
measurements are not available.

Given its widespread usage in related literature, occupant and room clustering
is calculated using the k-means clustering algorithm with Euclidean distance,
using the scikit-learn package888https://scikit-learn.org/stable/. The
features used for clustering were the ratio of votes of each feedback class
value for each subject. For example, the ratio of _prefer cooler_ for a given
participant, or room, would be calculated as follows: $\frac{\\#\text{prefer
cooler votes}}{\\#\text{total votes}}$. This calculation is repeated for all
types of feedback responses for thermal, light, and aural feedback. Then, the
number of clusters was chosen to match the number of possible responses per
type of feedback, this led to initially $k=9$, but given that there were no
data points with _prefer louder_ responses, the clusters were merged into
eight.

###  3.6 Occupant comfort preference prediction

The metric of comparison in this study was the prediction improvement of a
machine learning model using added feature sets extracted from the intensive
longitudinal preference data. This structure matches implementation-based
environmental comfort studies outlined in the literature that showed the
predictive improvement of additional data [64, 65]. This approach can be
compared to more controlled, lab-based methods that seek to isolate variables
and individually test their influence.

The prediction problem translates to predicting the right class value or, in
this case, the preference feedback response, at the given feature values. A
random forest classifier from the scikit learn package was chosen to handle
this comfort prediction. Random forest classifiers have been proven to have
the highest accuracy at predicting personal comfort in one previous study [77]
and is one of the best performing of other recent studies [64, 65, 78]. The
decision was made to focus on the implementation of a single model type that
has been proven effective and is straightforward to use based on documentation
and ease-of-tuning. With this in mind, we fixed the hyper-parameters for the
random forest classifier to 1000 number of trees, Gini criterion for node
splitting, and two minimum samples per split.

Additionally, the prediction problem was divided into an individual and a
grouped prediction task. The former refers to a model developed specifically
for a given occupant, using only parts of its data to train a model and test
it on its remaining data. On the other hand, the latter approach consists of
combining all occupants’ training data subsets with training a model and
testing it on all the occupants’ remaining data.

The data of each occupant was split into a 60:40 train test set based on time.
That is, the first 60% of votes from each occupant was used in their training
set, and the remaining 40% was used for testing. The sets were split by time
to prevent the scenario of future data being used to predict the past. For the
grouped model, all the occupants’ training sets (60% of each occupant’s data)
was used as one training set, and the remaining 40% of each occupant’s data
was combined with being used as one test set.

A primary component of the method was to test the ability for various feature
sets to influence the prediction power of the random forest model. The method
used six combinations of these feature sets to test the influence each has in
the predictive capability of the overall model. The following is an overview
of these feature categories developed for testing:

  * 1.

Time was created through feature engineering the time stamp of when an
occupant gave feedback. This feature was a cyclical representation of the hour
of the day and day of the week. This simple feature type detects if certain
cyclical habits or components have a role in preference prediction and was
included in all scenarios.

  * 2.

Environmental Sensors were features extracted from measurement data from
lighting (lux level), noise (dB level), temperature (deg. Celsius), and
relative humidity (RH%) measurement. These variables were collected from the
IEQ sensors that were closest spatially and temporally to an occupant when
they gave feedback.

  * 3.

Near Body Temperature was a feature created from the temperature sensor
mounted on the smartwatch strap that had temporal proximity to the time-stamp
of when the occupant gave feedback.

  * 4.

Heart Rate was collected from the Fitbit smartwatch device as an instantaneous
value collected when the occupant gave feedback.

  * 5.

Room was a feature that was encoded to a numerical preference type based on
the history of feedback in the room in which the survey was taken. This
feature was designed to increase the prediction accuracy by complimenting data
from rooms of similar comfort profiles. For example, if an occupant only works
from their office, the model will still be able to accurately predict how that
occupant may feel in other rooms that have a similar comfort profile to their
office.

  * 6.

Preference History features are similar to the Room features. These features
use the ratio of responses of each type (thermal, visual, and aural) that were
calculated for each user. This ratio was only calculated for the responses of
_prefer cooler_ , _prefer warmer_ , _prefer dimmer_ , _prefer brighter_ ,
_prefer quieter_ , and _prefer louder_. E.g., the ratio of response of _prefer
cooler_ responses of a given occupant is calculated the following way:
$\frac{\\#\text{prefer cooler votes}}{\\#\text{total votes}}$.

Model classification results were calculated using the F1-micro scores (as
shown in Equation 1) which were equivalent to accuracy in the a multi-class
classification problem by calculating precision and recall averaged across all
classes, i.e., subjective thermal comfort response value. As the objective was
to provide a comparison among different feature sets with a standard metric,
F1-micro was chosen due to its usage for benchmarking different aspects of the
modelling pipeline in thermal comfort datasets [78].

$F1=2\cdot\frac{precision\cdot recall}{precision+recall}$ (1)

Figure 4: Overview of the intensive longitudinal data collected from the
occupants according to the three categories: _Prefer warmer_ (red) and _Prefer
cooler_ (blue) for thermal, _Prefer Brighter_ (orange) and _Prefer Dimmer_
(purple) for lighting, and _Prefer Quieter_ (green) for acoustics. Each row is
an occupant and each box in that row shows that occupant’s feedback answers
collected sequentially. The visualisation is diagrammatic in that vertical
alignment of the boxes between different occupants does not imply identical
time stamps. Figure 5: K-means clustering of preference tendencies quantified
by average number of votes by occupant (a), and by room (b) within the test
building. Each row presents the percentage of votes that fell into a
respective preference. Dark colours in cells indicate higher preference.
Figure 6: Interactive visualisation of the data collection in the SDE4
building that highlights the spatial distribution of subjective preference
data for thermal comfort in three dimensions. In terms of preference feedback,
the blue dots indicate _prefer cooler_ responses, the yellow dots are _no
change_ , and the red dots are _prefer warmer_. Figure 7: Distribution of
sensor data by preference vote. While trends can be observed many feedback
votes overlap for the same environmental or physiological measurement. This
was possibly due to the different comfort tendencies as shown in Figure 5 or
numerous other variables described in Figure 1 that are not accounted for.
Near body temperature and noise appear to have the most distinct
differentiation.

##  4 Results

The results presented in this section are complemented with an interactive web
application999https://sde4demo.herokuapp.com/ and interactive
code101010https://github.com/buds-lab/humans-as-a-sensor-for-buildings which
enables the reader to regenerate all the plots. During a two week collection
time of 30 participants, 4,378 comfort preference votes were collected, which
is 146 data feedback points per person on average. From this set, 1,474 data
points were successfully localised to building environmental sensors. To allow
for comparison with those data, this subset was used for analysis and machine
learning in the following sections.

###  4.1 Grouping comfort preference tendencies

Figure 4 illustrates an overview of the intensive longitudinal preference
history data for each person according to the three preference categories.
These feedback responses were only those collected in the SDE4 building, and a
maximum number of 75 votes is shown. A simple clustering step was applied in
this figure to represent the segmentation according to each preference
category on its own. This visualisation shows how this simplified clustering
step captured the _tendency_ of an occupant to lean more towards one feedback
response over the others. This segmentation was independent of the
environmental parameters of the spaces to maintain the simplicity of the
approach. The subsequent modelling steps were designed to test the
effectiveness of doing this type of simplified segmentation.

Figure 5a is an aggregated representation of the segmentation process for each
occupant, this time with all three preference categories being used in the
clustering process. This figure summarises each occupant as a row of data, and
the colour of the box represents the percentage of votes given to a particular
preference category, where dark colours indicate higher preference. These
clusters provided segmentation of the users according to their preference
tendency types that were used in the preference models. Even in a sample size
of 30 occupants, there were varying comfort tendencies present, which
complemented the concept of a personal comfort model tested by Kim et al.
[79]. This clustering step provided the foundation for the creation of the
_individual versus grouped_ models used in the prediction step.

###  4.2 Tagging the spatial context with preference feedback

While the subjective feedback highlighted varying comfort tendencies within a
building, localisation also enabled the characterisation of preference
tendencies in certain zones. Figure 5b presents each room as a row, where the
colour of each cell represents the percentage of a preference vote given for a
particular room. The utilisation of k-means clustering once again enabled the
splitting and labelling of these zones, this time by the tendency for
different comfort preferences to be left by occupants in those spaces. This
result firstly served as an overview for facility managers to understand the
office spaces they manage, and take action to improve upon the comfort. A
visualisation of the subjective thermal preference data can be found in Figure
6 and online111111https://sde4demo.herokuapp.com/.

###  4.3 Correlation with indoor environmental quality variables

One standard aspect of environmental comfort studies is the comparison of
feedback to objective environmental measurements. For the data collected in
this study, standard distribution plots of the environmental sensor data are
summarised in Figure 7. Intuitive insight in the data can be observed, such as
the absence of _prefer brighter_ votes after an illuminance threshold of 250
lux. Nevertheless, there was a significant overlap between classes for each of
the environmental parameters, which were likely attributed to the numerous
unmeasured variables described by Figure 1, and the varying comfort tendencies
shown in Figure 5. This result reinforces the evidence that environmental
measurements are not descriptive enough to characterise a person’s
preferences, which results in poor prediction as found in previous studies
[50].

###  4.4 Predicting field-based indoor preference using intensive longitudinal
data

In this section, the time-series feedback was used to predict comfort
satisfaction. Figure 8 shows a comparison of the various models built with the
feature sets and process outlined in Section 3. The individual comfort model
uses the occupant’s training data for prediction, while the grouped comfort
model uses the input data for the groupings outlined in Figure 5. The top of
Figure 8 shows a table in which each row represents the feature set that was
used to train the model in that column.

Several insights were evident from this modelling analysis. Firstly, there
were only small differences in the F1 scores between the different feature
sets for the visual and aural preference models. These models, in general, had
higher F1 scores than thermal preference prediction. Aural preference
prediction had the highest F1 score, which was intuitive since it became a
binary classification challenge due to the lack of _prefer louder_ feedback
responses.

Thermal preference prediction had more diversity across the feature sets
tested as compared to the other preference categories. Merely using the
conventional time-series and environmental sensor features had the lowest F1
score. Adding the physiological attributes of heart rate and near body
temperature provided marginal improvements. The best thermal preference model
used the physiological, room, and preference history features while excluding
the environmental sensor data.

For all three preference categories, the grouped comfort model performed
better than the individual version. Participants with similar comfort
preferences became clustered together, thus increasing the training dataset
for that particular occupant type. This result showed the impact that
assigning a variety of _peer groups_ can have on preference prediction.

###  4.5 Cold-start comfort preference prediction

The success of the preference models using grouping allowed for models that
can predict an occupant’s preferences without their own personal data present.
This scenario was labelled as a _cold-start_ situation as it emulates when an
occupant doesn’t wear a watch to collect data in a particular building, but
relies on crowdsourced data from peers that have a similar comfort preference.
The line graphs to the right of Figure 8 show the results of this type of
analysis. They illustrate the number of occupants required to sufficiently
crowdsource the data for an average occupant for each of the preference
categories. The orange line represents an ordinary person who doesn’t wear a
smartwatch, whereas the blue line is a smartwatch owner who is regularly
giving feedback. In this study, nine and five users were sufficient on average
to crowdsource the thermal and visual comfort prediction respectively to the
same accuracy as a user wearing a watch.

Figure 8: Left: Comparison of prediction F1-micro-score between grouped and
individual comfort models using data from different feature sets. The feature
set that excluded environmental sensor data for the thermal model had the
highest F1-score, while minimal differences in F1-score were noted between
feature sets of the visual and aural models. Right: The accuracy in predicting
the comfort of an individual as further participants are added to the training
set. The blue line includes the test participants training set in the training
data, and the orange line excludes the test the participants training data
meaning that it depends on crowdsourced feedback from other occupants. Figure
9: Comfort prediction for two zones for an average occupant over a week. The
grey circles indicate votes that were given for each category, and the shaded-
out sections indicate times where no data were present. These time-series
predictions can be used to detect anomalies, such as the mid-day peak for a
warmer preference for the _office_ , or the general discomfort in the _outdoor
seating_ area. Note that there was an absence of data in these zones between
the hours of 22:00 - 7:00, and on the weekend. This lack of data caused
inaccurate predictions as seen in the square-shaped peaks in the _office_.

###  4.6 Predicting continuous comfort preference without sensors

Since the preference feedback in this methodology was at a much higher high-
frequency than a typical survey or occupants acting on the thermostat, this
study had preference data with relatively high temporal and spatial diversity.
The random forest classifier was used to predict comfort preference based on a
time-stamp input for each zone to create a continuous prediction over time.
This approach emulates the concept of using human feedback as a type of
sensor. Figure 9 illustrates the prediction of two different zones, an office
and an outdoor space, for a typical week using this model output. First, one
can see that the _office_ was generally a comfortable space, while the
_outdoor seating_ had an overall higher preference for cooling. Time-dependent
fluctuations show how the model was able to predict comfort preference for
different parts of the day or days of the week. The _office_ had a peak of
warmer preference around mid-day. Finally, it was observed how the model,
often inaccurately, tried to predict comfort at times where no data is
present. The square peaks in the _office_ for aural and visual prediction
between the hours of 22:00 and 7:00 were due to an absence of data to
accurately predict during these times.

##  5 Discussion

The results of this implementation showed the potential of collecting
intensive longitudinal feedback from occupants in the built environment. This
approach revealed that the deployment and implementation of such a methodology
were effective, and comfort models for visual, aural, and thermal comfort can
have similar performance to sensor measurements. The key focus in this section
is to discuss the practical uses and limitations of the proposed methodology.

###  5.1 Practical Application of Intensive Longitudinal Data in Industry

At the foundation of the method, the creation of more significant amounts of
occupant feedback information in the form of preferences was successful. The
utilisation of these type of high-frequency subjective feedback data has
potential for building evaluation and occupant comfort optimisation. It
changes the paradigm in which facility managers operate a building. For
example, instead of saying _light levels are below the comfort threshold in
Office-1_ , the new conclusions could state that _a higher frequency of prefer
brighter votes are recorded in Office-1_. Furthermore, due to the high-
frequency sampling rate provided by the micro ecological momentary assessment
methodology, these periods of discomfort can also be mapped to particular
times, and certain groups of people. The time series comfort profiles could
also serve as input data for occupant-centric-control efforts of building
systems which can then optimise for human comfort and energy optimisation.

####  5.1.1 Post-occupancy evaluation, commissioning, and sensor calibration

A focus on post-occupancy evaluation could be a key target for this type of
data collection. In this scenario, a particular sample of non-transient
occupants of a recently constructed building would be given smartwatches and
asked to wear them for 2-4 weeks. These data could then be used to supplement
the systems installed to characterise whether there are _blind spots_ in terms
of sensors not picking up comfort-influencing phenomenon that is not being
measured. For example, it’s rare to measure mean radiant temperature in most
buildings; therefore _hot spots_ might exist that are undetectable by
thermostats and might be a result of inadequate shading or control of shading
systems. To adapt the presented methodology to this context is straightforward
as the two-week time frame of the experiment is similar. The current method
involved asking each participant to wear the smartwatch until 100 data points
were recorded. In a real-world setting, a similar approach could be deployed.
At that point, perhaps the occupant could choose to return the device and rely
on the data of co-workers for comfort prediction or continue to wear it and
help crowdsource the data for others. It is strongly recommended that these
deployments use automated indoor localisation to put the feedback in the
spatial context without user intervention.

####  5.1.2 Potential for spatial recommendation systems and impact on
activity-based workspace design

A less typical application for intensive longitudinal data might be the
development of spatial recommendation engines for occupants in activity-based
workspaces. In these spaces, an occupant doesn’t have a constant workspace but
instead finds a space that matches their immediate needs. This paradigm could
prove to be an integral part of future working style, especially in light of
social distancing due to global pandemics such as COVID-19 that forces a less
conventional spatial working arrangement. This recommendation engine might
work in a way that an occupant’s comfort tendencies could be matched with the
comfort zone of the building. For example, those that prefer warmer spaces can
be recommended to work in areas that have a higher percentage of _prefer
cooler_ votes. Previous work in this direction showed progress using a
platform known as Spacematch [73]. Smart watch-based longitudinal feedback
could enhance the model development process for this type of platform.

The aspect of testing group-based models in this study is essential for this
context as building owners can’t expect all occupants to be willing to wear or
use devices. And those that do agree will likely have a limited amount of
patience for giving feedback over long periods. This paper tested the ability
to cluster occupants such that it was not necessary for everyone in an office
to wear a smartwatch. The only requirement for this type of system to work is
that each new employee would wear a smartwatch during a two-week data
collection phase, which is sufficient to build their comfort preference
tendency history as described in Section 4.1. The experiment also showed that
not everyone in an office space needed to be using the smartwatch application
all the time. In this particular experiment, six occupants were sufficient, on
average, to crowdsource the prediction for the remaining 24. This value is not
generalisable amongst all buildings and would change depending on the
different comfort tendencies the building occupants might present. The higher
the variation in comfort preferences, the greater the number of the occupants
needed to crowdsource the data.

In terms of office space design, the collection of intensive longitudinal
preference data could facilitate floor plan design decisions. Understanding
the breakdown of comfort needs according to the tendencies of the occupants
would enable architects to design or retrofit buildings with different comfort
zones to match the different types of people. For example, if the zones with
more cooling were popular and being used to their capacity, then the floor
plans or systems control could respond to by creating more spaces of that type
to increase the probability that a person feels comfortable.

####  5.1.3 Integration into building control systems

Intensive longitudinal data has the opportunity to influence the control and
automation systems of buildings through the use of preference feedback data in
the control logic. Most building control systems rely on optimising a set-
point temperature that is considered comfortable for the average occupant or
comfort standard [80]. In that scenario, discomfort instead of comfort is
evaluated as the current difference of the environment thermostat and the HVAC
system set-temperature [81], occupancy density estimation, or via more
traditional ways such as PMV [82]. While some of these approaches have dealt
with single-occupant offices or Personal Comfort Systems (PCS), there is a
distinction between controlling the actual HVAC system and allowing the
occupant to control their immediate space. PCS systems are those that locally
condition the occupant independent of the centralised HVAC system [83]. The
intensive longitudinal data and the models developed in this study could help
the controls field take the next step forward in occupant-centred building
controls through the use of reinforcement learning [84]. The feedback
mechanism in reinforcement control is generally the standard occupant-building
interface such as switch or thermostat [85]. Intensive longitudinal data could
be used to enhance that interaction by focusing on finding the motivations of
those control actions. This work is a strong focus of the occupant-centric
building operations in the IEA Annex 79 project [86].

###  5.2 Prediction models are only as good as the training data

The primary limitation of the presented approach was that it would only work
where data were present. As seen in Figure 9, there were errors in the
prediction when data were absent, i.e., no historical data collected at
similar time windows such as the middle of the night. Furthermore, since there
was a reliance on other subjects’ historical preferences, i.e., crowdsourcing
preferences, to evaluate environments, an office space that was rarely used
would have a poor prediction of occupant comfort. Classical comfort models
based on sensor data do not have this issue as spaces that are not used can
still be characterised by the measured data. Furthermore, this particular
study was conducted in Singapore, which doesn’t have seasons and has minimal
variability in temperature. For seasonal countries, the day of the year would
be an added feature that may take up to a year worth of data to train. Further
work could investigate the opportunity of using sensor data to characterise a
space and then continuously refine the comfort prediction by crowdsourcing the
occupants’ preference on said space.

##  6 Conclusions

This paper presents how micro ecological momentary assessments of subjective
comfort can generate sufficiently large intensive longitudinal data for
occupant comfort prediction and enhancement that can supplement objective
environmental sensor data, and empirical comfort models. Results of an
implementation of the platform on 30 occupants showed the segmentation and
variation of indoor occupant comfort tendencies and highlighted the
shortcomings of one-size-fits-all comfort models that are commonly applied in
real buildings. Furthermore, the use of a smartwatch enabled data collection
at a sufficient frequency to build time-series models of indoor spaces. These
models could be used to detect building anomalies, serve as building data for
subjective driven building control, or be used to recommend spaces that best
match the comfort preference tendencies of each occupant. The optimum
technological set-up uses a smartwatch for subjective data collection,
combined with a method for localising an occupant in the building. This
localisation may be achieved by asking the occupant directly through the
smartwatch, or through Bluetooth or WiFi signals.

## Author contribution statement

PJ: hardware, software, infrastructure development, experimental design,
implementation and lead author of the paper; MQ: software, infrastructure
development, data analysis, machine learning lead and author of the paper; MA:
software, infrastructure development, and author of the paper; CM: funding,
project leadership, experimental design, the corresponding author of the
paper.

## Funding

The Singapore Ministry of Education (MOE) (R296000181133 and R296000214114)
and the National University of Singapore (R296000158646) provided support for
the development and implementation of this research.

## Acknowledgements

This research contributes to the body of work for the International Energy
Agency (IEA) Energy in Building and Communities (EBC) Annex 79 - Occupant-
Centric Building Design and Operation.

## Data availability statement

Segments of the raw data and analysis code used for this study are available
in an open-access Github repository that includes further
documentation121212https://github.com/buds-lab/humans-as-a-sensor-for-
buildings.

## References

  * Frontczak et al. [2012] M. Frontczak, S. Schiavon, J. Goins, E. Arens, H. Zhang, P. Wargocki,  Quantitative relationships between occupant satisfaction and satisfaction aspects of indoor environmental quality and building design,  Indoor air 22 (2012) 119–131.
  * Sakellaris et al. [2016] I. Sakellaris, D. Saraga, C. Mandin, C. Roda, S. Fossati, Y. De Kluizenaar, P. Carrer, S. Dimitroulopoulou, V. Mihucz, T. Szigeti, et al.,  Perceived indoor environment and occupants’ comfort in european “modern” office buildings: The officair study,  International journal of environmental research and public health 13 (2016) 444\.
  * Fanger et al. [1970] P. O. Fanger, et al.,  Thermal comfort. analysis and applications in environmental engineering.,  Thermal comfort. Analysis and applications in environmental engineering. (1970).
  * De Dear and Brager [1998] R. De Dear, G. S. Brager,  Developing an adaptive model of thermal comfort and preference,  ASHRAE Transactions, Vol 104 (1998).
  * Asadi et al. [2016] I. Asadi, I. Hussein, K. Palanisamy, W. Conference, M. Comfort, R. Cumberland, E. Use, C. Environmental, M. Engineering, V. Mesiano, R. Sulaiman, S. N. Kamaruzzaman, T. Ramos, S. Dedesko, J. A. Siegel, J. A. Gilbert, B. Stephens, I. Asadi, N. Mahyuddin, P. Shafigh, E. M. d. A. Vieira, L. B. da Silva, E. L. de Souza,  A survey of evaluation methods used for holistic comfort assessment,  PLoS ONE 953-954 (2016) 1513–1519.
  * Tang et al. [2020] H. Tang, Y. Ding, B. Singer,  Interactions and comprehensive effect of indoor environmental quality factors on occupant satisfaction,  Building and Environment 167 (2020).
  * Henry and Emery [1986] C. Henry, B. Emery,  Effect of spiced food on metabolic rate.,  Human nutrition. Clinical nutrition 40 (1986) 165–168.
  * Swaminathan et al. [1985] R. Swaminathan, R. King, J. Holmfield, R. Siwek, M. Baker, J. Wales,  Thermic effect of feeding carbohydrate, fat, protein and mixed meal in lean and obese subjects,  The American journal of clinical nutrition 42 (1985) 177–181.
  * Webster et al. [2007] T. Webster, F. Bauman, G. Anwar,  CBE Portable Wireless Monitoring System (PWMS): UFAD Systems Commissioning Cart Design Specifications and Operating Manual,  Internal Report, Center for the Built Environment, UC Berkeley (2007) 4.
  * Choi et al. [2012] J. H. Choi, V. Loftness, A. Aziz,  Post-occupancy evaluation of 20 office buildings as basis for future IEQ standards and guidelines,  in: Energy and Buildings, volume 46, pp. 167–175.
  * Kim [2013] H. Kim,  Methodology for Rating a Building’s Overall Performance based on the ASHRAE/CIBSE/USGBC Performance Measurement Protocols for Commercial Buildings,  Journal of Chemical Information and Modeling 53 (2013) 1689–1699.
  * Parkinson et al. [2019] T. Parkinson, A. Parkinson, R. de Dear,  Continuous IEQ monitoring system: Context and development,  Building and Environment 149 (2019) 15–25.
  * Nicol and Wilson [2011] J. F. Nicol, M. Wilson,  A critique of European Standard EN 15251: Strengths, weaknesses and lessons for future standards,  Building Research and Information 39 (2011) 183–193.
  * Schiavon et al. [2017] S. Schiavon, B. Yang, Y. Donner, V.-C. Chang, W. W. Nazaroff,  Thermal comfort, perceived air quality, and cognitive performance when personally controlled air movement is used by tropically acclimatized persons,  Indoor air 27 (2017) 690–702.
  * Hodder and Parsons [2007] S. G. Hodder, K. Parsons,  The effects of solar radiation on thermal comfort,  International journal of biometeorology 51 (2007) 233–250.
  * Kräuchi [2002] K. Kräuchi,  How is the circadian rhythm of core body temperature regulated?,  Clinical Autonomic Research 12 (2002) 147–149.
  * Chinazzo et al. [2019] G. Chinazzo, J. Wienold, M. Andersen,  Daylight affects human thermal perception,  Scientific reports 9 (2019) 1–15.
  * Halawa et al. [2014] E. Halawa, J. van Hoof, V. Soebarto,  The impacts of the thermal radiation field on thermal comfort, energy consumption and control—a critical overview,  Renewable and Sustainable Energy Reviews 37 (2014) 907–918.
  * Fukazawa and Havenith [2009] T. Fukazawa, G. Havenith,  Differences in comfort perception in relation to local and whole body skin wettedness,  European journal of applied physiology 106 (2009) 15–24.
  * Kingma et al. [2012] B. Kingma, A. Frijns, W. van Marken Lichtenbelt,  The thermoneutral zone: implications for metabolic studies,  Front Biosci (Elite Ed) 4 (2012) 1975–1985.
  * Tikuisis and Ducharme [1996] P. Tikuisis, M. B. Ducharme,  The effect of postural changes on body temperatures and heat balance,  European journal of applied physiology and occupational physiology 72 (1996) 451–459.
  * Gagge et al. [1967] A. P. Gagge, J. Stolwijk, J. Hardy,  Comfort and thermal sensations and associated physiological responses at various ambient temperatures,  Environmental research 1 (1967) 1–20.
  * Gaesser and Brooks [1975] G. A. Gaesser, G. A. Brooks,  Muscular efficiency during steady-rate exercise: effects of speed and work rate,  Journal of applied physiology 38 (1975) 1132–1139.
  * Havenith et al. [2015] G. Havenith, K. Kuklane, J. Fan, S. Hodder, Y. Ouzzahra, K. Lundgren, Y. Au, D. L. Loveday,  A database of static clothing thermal insulation and vapor permeability values of non-western ensembles for use in ashrae standard 55, iso 7730, and iso 9920 ch-15-018 (rp-1504) (2015).
  * Johnson and Kellogg Jr [2010] J. M. Johnson, D. L. Kellogg Jr,  Thermoregulatory and thermal control in the human cutaneous circulation,  Front Biosci (Schol Ed) 2 (2010) 825–853.
  * Zhang and Hedge [2014] H. Zhang, A. Hedge,  Laptop heat and models of user thermal discomfort,  in: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 58, SAGE Publications Sage CA: Los Angeles, CA, pp. 1456–1460.
  * Karjalainen [2012] S. Karjalainen,  Thermal comfort and gender: a literature review,  Indoor air 22 (2012) 96–109.
  * Huang et al. [2010] H.-W. Huang, W.-C. Wang, C.-C. K. Lin,  Influence of age on thermal thresholds, thermal pain thresholds, and reaction time,  Journal of Clinical Neuroscience 17 (2010) 722–726.
  * Havenith et al. [2002] G. Havenith, I. Holmér, K. Parsons,  Personal factors in thermal comfort assessment: clothing properties and metabolic heat production,  Energy and buildings 34 (2002) 581–591.
  * Frank et al. [1999] S. M. Frank, S. N. Raja, C. F. Bulcao, D. S. Goldstein,  Relative contribution of core and cutaneous temperatures to thermal comfort and autonomic responses in humans,  Journal of Applied physiology 86 (1999) 1588–1593.
  * Boulant [2000] J. A. Boulant,  Role of the preoptic-anterior hypothalamus in thermoregulation and fever,  Clinical infectious diseases 31 (2000) S157–S161.
  * Cabanac [1971] M. Cabanac,  Physiological role of pleasure,  Science 173 (1971) 1103–1107.
  * Brainard et al. [2001] G. C. Brainard, J. P. Hanifin, J. M. Greeson, B. Byrne, G. Glickman, E. Gerner, M. D. Rollag,  Action spectrum for melatonin regulation in humans: evidence for a novel circadian photoreceptor,  Journal of Neuroscience 21 (2001) 6405–6412.
  * Perez et al. [1990] R. Perez, P. Ineichen, R. Seals, J. Michalsky, R. Stewart,  Modeling daylight availability and irradiance components from direct and global irradiance,  Solar energy 44 (1990) 271–289.
  * Leather et al. [1998] P. Leather, M. Pyrgas, D. Beale, C. Lawrence,  Windows in the workplace: Sunlight, view, and occupational stress,  Environment and behavior 30 (1998) 739–762.
  * Dai et al. [2016] Q. Dai, L. Hao, Y. Lin, Z. Cui,  Spectral optimization simulation of white light based on the photopic eye-sensitivity curve,  Journal of Applied Physics 119 (2016) 053103.
  * Slater and Boyce [1990] A. I. Slater, P. R. Boyce,  Illuminance uniformity on desks: Where is the limit?,  Lighting research & technology 22 (1990) 165–174.
  * Baron et al. [1992] R. A. Baron, M. S. Rea, S. G. Daniels,  Effects of indoor lighting (illuminance and spectral distribution) on the performance of cognitive tasks and interpersonal behaviors: The potential mediating role of positive affect,  Motivation and emotion 16 (1992) 1–33.
  * Wilkins et al. [1989] A. J. Wilkins, I. Nimmo-Smith, A. I. Slater, L. Bedocs,  Fluorescent lighting, headaches and eyestrain,  Lighting Research & Technology 21 (1989) 11–18.
  * Main et al. [1997] A. Main, A. Dowson, M. Gross,  Photophobia and phonophobia in migraineurs between attacks,  Headache: The Journal of Head and Face Pain 37 (1997) 492–495.
  * Yin et al. [2018] J. Yin, S. Zhu, P. MacNaughton, J. G. Allen, J. D. Spengler,  Physiological and cognitive performance of exposure to biophilic indoor environment,  Building and Environment 132 (2018) 255–262.
  * Hopkinson [1972] R. G. Hopkinson,  Glare from daylighting in buildings,  Applied ergonomics 3 (1972) 206–215.
  * Pierrette et al. [2015] M. Pierrette, E. Parizet, P. Chevret, J. Chatillon,  Noise effect on comfort in open-space offices: development of an assessment questionnaire,  Ergonomics 58 (2015) 96–106.
  * Job [1988] R. Job,  Community response to noise: A review of factors influencing the relationship between noise exposure and reaction,  The Journal of the Acoustical Society of America 83 (1988) 991–1001.
  * Kim and De Dear [2013] J. Kim, R. De Dear,  Workspace satisfaction: The privacy-communication trade-off in open-plan offices,  Journal of Environmental Psychology 36 (2013) 18–26.
  * Templeton and Saunders [2014] D. Templeton, D. Saunders, Acoustic design, Elsevier, 2014\.
  * Lee and Brand [2010] S. Y. Lee, J. Brand,  Can personal control over the physical environment ease distractions in office workplaces?,  Ergonomics 53 (2010) 324–335.
  * Kjellberg and Sköldström [1991] A. Kjellberg, B. Sköldström,  Noise annoyance during the performance of different nonauditory tasks,  Perceptual and motor skills 73 (1991) 39–49.
  * Banbury and Berry [2005] S. P. Banbury, D. C. Berry,  Office noise and employee concentration: Identifying causes of disruption and potential improvements,  Ergonomics 48 (2005) 25–37.
  * Cheung et al. [2019] T. Cheung, S. Schiavon, T. Parkinson, P. Li, G. Brager,  Analysis of the accuracy on pmv–ppd model using the ashrae global thermal comfort database ii,  Building and Environment (2019).
  * Stone et al. [2007] A. A. Stone, S. Shiffman, A. A. Atienza, L. Nebeling, A. Stone, S. Shiffman, A. Atienza, L. Nebeling,  Historical roots and rationale of ecological momentary assessment (ema),  The science of real-time data capture: Self-reports in health research (2007) 3–10.
  * Intille et al. [2016] S. Intille, C. Haynes, D. Maniar, A. Ponnada, J. Manjourides,  $\mu$ema: Microinteraction-based ecological momentary assessment (ema) using a smartwatch,  in: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, ACM, pp. 1124–1128.
  * Wang et al. [2014] D. Wang, M. T. Amin, S. Li, T. Abdelzaher, L. Kaplan, S. Gu, C. Pan, H. Liu, C. C. Aggarwal, R. Ganti, X. Wang, P. Mohapatra, B. Szymanski, H. Le,  Using humans as sensors: an estimation-theoretic perspective,  in: Proceedings of the 13th international symposium on Information processing in sensor networks, IPSN ’14, IEEE Press, 2014, pp. 35–46.
  * Avvenuti et al. [2016] M. Avvenuti, M. G. C. A. Cimino, S. Cresci, A. Marchetti, M. Tesconi,  A framework for detecting unfolding emergencies using humans as sensors,  Springerplus 5 (2016) 43.
  * Vielberth et al. [2019] M. Vielberth, F. Menges, G. Pernul,  Human-as-a-security-sensor for harvesting threat intelligence,  Cybersecurity 2 (2019) 23.
  * Heinzerling et al. [2013] D. Heinzerling, S. Schiavon, T. Webster, E. Arens,  Indoor environmental quality assessment models: A literature review and a proposed weighting and classification scheme 70 (2013) 210–222.
  * Ncube and Riffat [2012] M. Ncube, S. Riffat,  Developing an indoor environment quality tool for assessment of mechanically ventilated office buildings in the UK - A preliminary study,  Building and Environment 53 (2012) 26–33.
  * Wong et al. [2008] L. T. Wong, K. W. Mui, P. S. Hui,  A multivariate-logistic model for acceptance of indoor environmental quality (IEQ) in offices,  Building and Environment 43 (2008) 1–6.
  * Lai et al. [2009] A. C. Lai, K. W. Mui, L. T. Wong, L. Y. Law,  An evaluation model for indoor environmental quality (IEQ) acceptance in residential buildings,  Energy and Buildings 41 (2009) 930–936.
  * Cohen et al. [2001] R. Cohen, M. Standeven, B. Bordass, A. Leaman,  Assessing building performance in use 1: The Probe process,  Building Research and Information 29 (2001) 85–102.
  * Webster et al. [2007] T. Webster, E. Arens, G. Anwar, J. Bonnell, F. Bauman, C. Brown,  Ufad commissioning cart: Design specifications and operating manual (2007).
  * Jin et al. [2018] M. Jin, S. Liu, S. Schiavon, C. Spanos,  Automated mobile sensing: Towards high-granularity agile indoor environmental quality monitoring,  Building and Environment 127 (2018) 268–276.
  * Porter et al. [2004] S. R. Porter, M. E. Whitcomb, W. H. Weitzer,  Multiple surveys of students and survey fatigue,  New Directions for Institutional Research 2004 (2004) 63–73.
  * Liu et al. [2019] S. Liu, S. Schiavon, H. P. Das, M. Jin, C. J. Spanos,  Personal thermal comfort models with wearable sensors,  Build. Environ. 162 (2019) 106281.
  * Aryal and Becerik-Gerber [2019] A. Aryal, B. Becerik-Gerber,  A comparative study of predicting individual thermal sensation and satisfaction using wrist-worn temperature sensor, thermal camera and ambient temperature sensor,  Build. Environ. 160 (2019) 106223.
  * Clear et al. [2018] A. K. Clear, S. Mitchell Finnigan, P. Olivier, R. Comber,  ThermoKiosk: Investigating Roles for Digital Surveys of Thermal Experience in Workplace Comfort Management,  Proc. of CHI (2018) 1–12.
  * Engelen and Held [2019] L. Engelen, F. Held,  Understanding the office: Using ecological momentary assessment to measure activities, posture, social interactions, mood, and work performance at the workplace,  Buildings 9 (2019) 54.
  * Monnot et al. [2016] B. Monnot, E. Wilhelm, G. Piliouras, Y. Zhou, D. Dahlmeier, H. Y. Lu, W. Jin,  Inferring activities and optimal trips: Lessons from singapore’s national science experiment,  in: Complex Systems Design & Management Asia, Springer International Publishing, 2016, pp. 247–264.
  * Wilhelm et al. [2016] E. Wilhelm, Y. Zhou, N. Zhang, J. Kee, G. Loh, N. Tippenhauer, Sensg: Large-scale deployment of wearable sensors for trip and transport mode logging, Technical Report, 2016\.
  * Benita et al. [2019] F. Benita, G. Bansal, B. Tunçer,  Public spaces and happiness: Evidence from a large-scale field experiment,  Health Place 56 (2019) 9–18.
  * Ojha et al. [2019] V. K. Ojha, D. Griego, S. Kuliga, M. Bielik, P. Buš, C. Schaeben, L. Treyer, M. Standfest, S. Schneider, R. König, D. Donath, G. Schmitt,  Machine learning approaches to understand the influence of urban environments on human’s physiological response,  Inf. Sci. 474 (2019) 154–169.
  * Rahaman et al. [2020] M. S. Rahaman, J. Liono, Y. Ren, J. Chan, S. Kudo, T. Rawling, F. D. Salim,  An Ambient-Physical system to infer concentration in open-plan workplace,  IEEE Internet of Things Journal (2020) 1–1.
  * Sood et al. [2020] T. Sood, P. Janssen, C. Miller,  Spacematch: Using environmental preferences to match occupants to suitable activity-based workspaces,  Frontiers in Built Environment 6 (2020) 113.
  * Sood et al. [2019] T. Sood, M. Quintana, P. Jayathissa, M. AbdelRahman, C. Miller,  The SDE4 learning trail: Crowdsourcing occupant comfort feedback at a net-zero energy building,  J. Phys. Conf. Ser. 1343 (2019) 012141.
  * Jayathissa et al. [2019] P. Jayathissa, M. Quintana, T. Sood, N. Narzarian, C. Miller,  Is your clock-face cozie? a smartwatch methodology for the in-situ collection of occupant comfort data,  CISBAT - Climate Resilient Cities - Energy Efficiency and Renewables in the Digital Era (2019).
  * Abdelrahman et al. [2019] M. M. Abdelrahman, P. Jayathissa, C. Miller,  YAK: An Indoor Positioning App for Spatial-Temporal Indoor Environmental Quality Research (2019).
  * Kim et al. [2018] J. Kim, Y. Zhou, S. Schiavon, P. Raftery, G. Brager,  Personal comfort models: predicting individuals’ thermal preference using occupant heating and cooling behavior and machine learning,  Building and Environment 129 (2018) 96–106.
  * Luo et al. [2020] M. Luo, J. Xie, Y. Yan, Z. Ke, P. Yu, Z. Wang, J. Zhang,  Comparing machine learning algorithms in predicting thermal sensation using ASHRAE Comfort Database II,  Energy and Buildings 210 (2020) 109776.
  * Kim et al. [2018] J. Kim, S. Schiavon, G. Brager,  Personal comfort models–a new paradigm in thermal comfort for occupant-centric environmental control,  Building and Environment 132 (2018) 114–124.
  * Enescu [2017] D. Enescu,  A review of thermal comfort models and indicators for indoor environments,  Renewable and Sustainable Energy Reviews 79 (2017) 1353–1379.
  * Barrios and Kleiminger [2017] L. Barrios, W. Kleiminger,  The Comfstat - Automatically sensing thermal comfort for smart thermostats,  2017 IEEE International Conference on Pervasive Computing and Communications, PerCom 2017 (2017) 257–266.
  * Park and Nagy [2018] J. Y. Park, Z. Nagy,  Comprehensive analysis of the relationship between thermal comfort and building control research - A data-driven literature review,  Renewable and Sustainable Energy Reviews 82 (2018) 2664–2679.
  * Zhang et al. [2015] H. Zhang, E. Arens, Y. Zhai,  A review of the corrective power of personal comfort systems in non-neutral ambient environments,  Building and Environment 91 (2015) 15–41.
  * Park et al. [2019a] J. Y. Park, M. M. Ouf, B. Gunay, Y. Peng, W. O’Brien, M. B. Kjærgaard, Z. Nagy,  A critical review of field implementations of occupant-centric building controls,  Build. Environ. 165 (2019a) 106351.
  * Park et al. [2019b] J. Y. Park, T. Dougherty, H. Fritz, Z. Nagy,  LightLearn: An adaptive and occupant centered controller for lighting based on reinforcement learning,  Build. Environ. 147 (2019b) 397–414.
  * O’Brien et al. [2020] W. O’Brien, A. Wagner, M. Schweiker, A. Mahdavi, J. Day, M. B. Kjærgaard, S. Carlucci, B. Dong, F. Tahmasebi, D. Yan, T. Hong, H. B. Gunay, Z. Nagy, C. Miller, C. Berger,  Introducing IEA EBC annex 79: Key challenges and opportunities in the field of occupant-centric building design and operation,  Build. Environ. 178 (2020) 106738.
  * Duarte Roa et al. [2020] C. Duarte Roa, S. Schiavon, T. Parkinson,  Targeted occupant surveys : A novel method to effectively relate occupant feedback with environmental conditions,  Preprint (2020).
  * Lipczynska et al. [2018] A. Lipczynska, S. Schiavon, L. T. Graham,  Thermal comfort and self-reported productivity in an office with ceiling fans in the tropics,  Building and Environment 135 (2018) 202–212.
  * Quintana and Miller [2019] M. Quintana, C. Miller,  Poster Abstract: Towards Class-Balancing Human Comfort Datasets with GANs,  in: BuildSys ’19 Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Built Environments, New York, NY, USA.
  * Abdelrahman et al. [2020] M. M. Abdelrahman, A. Chong, C. Miller,  Build2Vec: Building Representation in Vector Space,  in: SimAUD 2020, Online, pp. 101–104.