 41. What Do We Know When We Know Ourselves? 841
practical for researchers and participants. 
We typically cannot observe people’s behav -
ior in the laboratory for more than a few 
hours for ethical and practical reasons (e.g., 
expensive and labor-  intensive). Naturalistic 
observation, on the other hand, is easier on 
the participants and researchers, because 
participants give consent to be observed and 
can go about their everyday lives. This en -
ables researchers to observe participants for 
longer periods of time across a wider range 
of situations. Third, rather than relying on 
one instance of laboratory-  based behavior, 
naturalistic observations provide many ob -
servations across situations per participant, 
making the criterion measures more valid.
The downsides of naturalistic behavior 
observation are not trivial. We cannot un -
obtrusively track people in their natural 
environments for days on end. Recording a 
person’s every move is impossible and often 
unethical. Instead, researchers typically sam -
ple thin slices of people’s everyday behavior. 
One way to do so unobtrusively and to pro -
tect the privacy of participants’ interlocutors 
is to record only the acoustic channel—  to 
sample audio snippets of people’s daily lives. 
The electronically activated recorder (EAR; 
Mehl, Pennebaker, Crow, Dabbs, & Price, 
2001) allows the researcher to do this. The 
EAR is a free application that runs on Apple 
devices (e.g., iPod Touch) and can be pro -
grammed to turn the audio recording func -
tion on and off at regular intervals, although 
the participant does not know when the re -
corder is on. In typical EAR research, par -
ticipants are given a locked iPod Touch in 
a case with a clip, which can be attached to 
the outside of participants’ clothing or hand -
bag (e.g., backpack, purse, or briefcase). Past 
work suggests that even 2–3 days of record -
ing can provide rich, valid data about rela -
tively common and acoustically detectable 
behavior (e.g., talking, moral behaviors; 
Bollich et al., 2016). Mehl and his colleagues 
(2001) have written extensively about the 
legal, ethical and practical issues involved 
with EAR research (e.g., Manson & Rob -
bins, 2017; Robbins, 2017), and these vary 
quite a bit by jurisdiction, population under 
study, and goals of the study.
The greatest downside to naturalistic be -
havioral observation is the time and person -
nel it takes to code the large amounts of behavioral data. These data are incredibly 
rich, which means dozens of behaviors can 
be coded at different levels of specificity. 
Going further, “macro” behaviors (e.g., sar -
casm, humor, arrogance) require some inter -
pretation on the part of coders, and other 
sources of error also reduce the “objectivity” 
of behavior coding (Rentfrow & Gosling, 
Chapter 40, this volume). Thus, it is advis -
able to have each recording coded by least 
two and preferably three or more coders to 
yield reliable ratings. A study using the EAR 
might yield 600 files of 30 seconds for each 
participant, and as such, requires hundreds 
of hours of coding by each coder.
A final downside to observational behav -
ioral data is that these data can only tell us 
how the participant was acting and not what 
the participant was thinking or feeling. For 
example, if Jane was coded as gossiping, it 
is unclear whether her intent was to protect 
others from an immoral person or to hurt 
someone’s reputation. In order to evaluate 
the accuracy of self-  reports of typical pat -
terns of thinking or feeling (e.g., self-  reports 
of anxiety, optimism, self-  esteem), other 
criterion measures are necessary. For these 
internal attributes, it may be impossible to 
avoid the circularity of evaluating the accu -
racy of self-  reports with other self-  reports. 
For example, if we want to know whether 
a person is aware of her typical levels of 
positive affect, we might have to compare 
her global self-view to her momentary 
self- reports of state affect assessed repeat -
edly using the experience sampling method 
(ESM; e.g., Spain, Eaton, & Funder, 2000).
A third type of behavioral criterion is an 
objective life outcome, such as grade point 
average (GPA), health, or divorce. These out -
comes are believed to represent the conse -
quences of many aggregated behaviors over 
time that stem from personality (Roberts, 
Kuncel, Shiner, Caspi, & Goldberg, 2007). 
For example, conscientiousness should pre -
dict a wide range of health behaviors (e.g., 
wearing sunscreen) that in turn, lead to 
fewer health problems. As such, the degree 
to which self-  reports of personality (e.g., 
conscientiousness) predict these outcomes 
(e.g., skin cancer) might indicate the degree 
to which people have self-  knowledge.
In short, the criterion problem is a thorny 
one. The best solution is often triangula -