compared to other traits. Thus, inter-individual comparisons with regard to the absolute level of a trait are not possi-
ble. This would make forced choice questionnaires useless in most applied settings where selection or the compari-
son to an absolute standard is important. However, more recent approaches have grouped four items from different
domains and asked participants to choose the alternatives most and least like them. These formats are often called
multidimensional pairwise preference scales ( Chernyshenko et al., 2009 ). Using item response based approaches
(Brown & Maydeu-Olivares, 2011; Stark et al., 2012 ), it is now not only possible to extract a normative score allowing
for inter-individual comparisons, but empirical evidence also suggest that such scales indeed are more immune to
social desirable responding, especially intentional distortion. However, constructing such scales is an immensely
demanding enterprise. Moreover, effects of coaching test takers have not been tested yet. Nevertheless, this
approach has great potential to overcome some of the problems discussed so far.
A less costly approach involves so called anchoring vignettes ( King & Wand, 2007 ). This technique requires
the administration of additional items at the beginning of a questionnaire. At this point, participants are asked to
rate the descriptions of hypothetical people using the questionnaire’s rating scale. These described profiles have
already been rated by experts. Using this information, the participants’ answers are used as anchors for their self-
descriptions. Applying an intuitive formula, the scores are then transformed to correct for response biases.
Anchoring vignettes have successfully been applied in large scale assessments such as PISA ( Buckley, 2009 ).
However, as becomes evident from their description, although anchoring vignettes may demand less investment
than multidimensional pairwise preference scales, they are by no means easy to develop. More empirical evi-
dence is needed before any further conclusions can be drawn.
OTHER ATTEMPTS TO DETECT SOCIALLY DESIRABLE RESPONDING
Within the preceding parts of the chapter a detailed overview of attempts to detect social desirable responding
using some form of questionnaire has been provided. Other attempts to detect response distortion also exist in a
wide variety.
Ziegler and Bu ¨hner (2009) introduced a method in which they combined a known group experimental design
and structural equation techniques to model a latent variable comprising variance due to response distortion.
Analysing data from a sample collected in a laboratory setting, the authors demonstrated that social desirable
responding affects the internal structure of a personality questionnaire but not the test-criterion correlations of
the test scores. However, whereas the approach might be useful to model faking, its use to identify individual
response distortion seems limited. An additional problem is the need to collect data derived from an anonymous
situation for the same participants. This caveat also applies to the otherwise promising approach suggested by
Griffith et al. (2007) in which they used confidence intervals around honest scores to detect response distortion.
For several years now, the idea of applying item response techniques to capture individual response distortion
has attracted a lot of attention. Most notably, Zickar and colleagues ( Zickar, Gibby & Robie 2004; Zickar & Robie,
1999; Zickar & Sliter, 2012 ) have applied the so called mixed Rasch model to identify response distortions on an
individual level. These models look for qualitatively distinct response classes based on distinctive response pat-
terns. Within each response class, the trait scores are allowed to vary. That way, mixed Rasch models are a com-
bination of latent class analysis and Rasch models ( Rost, 1991; Rost, Carstensen & Von Davier 1997 ). Based on
this approach, Zickar and colleagues have described two response classes supposedly representing slight and
extreme faking. This otherwise promising approach has been critically viewed by other researchers. One of the
criticisms is the conceptual similarity with midpoint and extreme point responding, two response biases unre-
lated to response distortion ( Ziegler & Kemper, 2013 ). Thus, further studies are needed to determine the specific-
ity of mixed Rasch models to identify individual response distortion. A similar approach but based on a
combination of structural equation modeling and latent class analysis has been used by Leite (2010) .
A statistically less demanding approach to identify response distortion is the use of decision trees ( Lukoff,
2012; Lukoff, Heggestad, Kyllonen & Roberts 2007 ). This approach first requires data from a known group
design. Using learning algorithms, decision trees are generated. Such trees state the probability of a person dis-
torting the answer when choosing a certain response category. These probabilities were gathered from the data.
The approach is relatively intuitive and can be implemented in computer-based assessment. It is also possible to
combine this technique with warnings that are shown to the test taker during the test session if certain response
categories or category combinations across items are chosen. Despite the relative ease of the technique, empirical
findings regarding actual sensitivity and specificity of the trees to discriminate between honest and distorted
responding reveal further need for improvement.37 OTHER ATTEMPTS TO DETECT SOCIALLY DESIRABLE RESPONDING
I. INTRODUCTION: CORE ISSUES IN ASSESSMENT