840 vIII. EMERGING  ISSuES AND  NEW  DIRECTIONS
Methodological Issues 
with Assessing Accuracy
The Criterion  Problem
Studying accuracy in any context is challeng -
ing. For the enterprise to make sense, accu -
racy research entails comparing some judg -
ment (e.g., Keiko’s perception of Amar) to a 
criterion measure that is more valid than the 
judgment or that at least captures aspects of 
the truth not captured by the judgment. This 
is challenging enough when the judgment is 
from a nonexpert—  if Keiko does not know 
Amar very well, then there are a number of 
possible criterion measures available against 
which to evaluate the accuracy of Keiko’s 
judgment, including Amar’s self-  rating, rat -
ings of Amar by his friends, and so forth. 
However, when the judgment is already like -
ly to be quite valid, such as a self-  judgment 
(i.e., when we want to know the accuracy of 
Amar’s self-view), it becomes extraordinari -
ly hard to identify a good criterion measure.
There are several approaches researchers 
can take when attempting to evaluate the ac -
curacy of self-views. The first is to use the 
aggregated judgments of close others as the 
criterion, relying on the notion that close 
others know things about us that we don’t 
know about ourselves and their consensual 
judgments reflect some kind of social reality. 
This approach is better for external aspects 
of personality (e.g., moral behavior) and at -
tributes that are inherently reputational in 
nature (e.g., being funny) than for internal 
attributes (e.g., thoughts, feelings, or aspira -
tions). Some of our own research, described 
in greater detail below, suggests that close 
others are likely quite accurate and some -
times more accurate than the self (Vazire, 
2010; Vazire & Carlson, 2010, 2011; Vazire 
& Mehl, 2008).
Another approach is to use behavioral 
tests as a criterion measure. Researchers can 
create situations, typically in a laboratory 
setting, designed to elicit individual differ -
ences in behavior that reflects people’s per -
sonality in a given domain. For example, the 
leaderless group discussion task (Bass, 1954) 
has been used to observe people’s dominance 
and talkativeness, the Trier Social Stress 
Test (TSST; Kirschbaum, Pirke, & Hellham -
mer, 1993) has been used to observe people’s 
social anxiety, and the Brick Creativity Test (Friedman & Förster, 2002) has been used 
to observe creativity. These behavioral tests 
are meant to capture manifestations of un -
derlying traits by putting people in a situa -
tion in which their reaction likely depends 
on their trait level (e.g., a socially anxious 
person should react very differently to the 
TSST than someone low on social anxiety). 
These tests can be quite powerful if designed 
well and can elicit diagnostic behavior that 
is unlikely to occur in the normal course of 
a person’s day or week. Moreover, by expos -
ing everyone to the same situation, we can be 
more confident that differences in behavior 
are due to differences in personality. How -
ever, a major downside to behavioral tests 
is that they are typically single-  shot mea -
sures of behavior. As we have learned from 
the person–  situation debate (Epstein, 1979), 
behavioral measures are notoriously unreli -
able, and like most personality measures, 
a valid behavioral measure of personality 
typically requires aggregating across many 
instances of behaviors (i.e., a multiple-  item 
measure).
A third approach is to capture naturally 
occurring behavior in people’s daily lives; 
that is, rather than observe someone in a 
contrived setting in the laboratory, research -
ers record momentary assessments of be -
havior as it naturally occurs in daily life. 
This approach has several advantages. First, 
this criterion likely matches what people 
have in mind when they describe their own 
personalities—  they are trying to describe 
what they are typically like in their actual 
lives, not in hypothetical situations in which 
they may never find themselves. For exam -
ple, Julia may describe herself as relatively 
agreeable, even though she knows she is dis -
agreeable toward people outside her inner 
circle of friends. Knowing this about herself, 
Julia is careful to interact mostly with peo -
ple close to her; thus, her behavior is mostly 
agreeable. When Julia describes herself as 
very agreeable, she is thinking of her actual 
day-to-day life, which involves interacting 
mostly with her close friends. By observing 
Julia’s behavior in her natural environments 
rather than placing her in a new environ -
ment she would not have selected (e.g., inter -
acting with strangers), we can more directly 
test whether Julia’s description of herself is 
accurate. Second, naturalistic behavior is 