On average, the chance of passing a driving test at the first attempt is about _______ in 100.
The chances that the problem of terrorism will be solved are about _______ in 100.
The chance that if you put your mind into something, your goals would come true is about ______ in 100.
The chances that virtual reality will become the main entertainment in the future are about _______ in 100.
The probability that the human race will survive for another thousand years is about _____ in 100.
The chance that if you put your mind and effort into solving a problem, you would succeed is about _______
in 100.
The chance that if you open a business it would succeed is about _______ in 100.
CALIBRATION AND THE JUDGMENT OF ACCURACY
Current interest in confidence ratings was motivated by a desire to compare confidence and accuracy of per-
formance. This comparison can be expressed either graphically, in the form of calibration curves, or using several
different scoring formulas ( Stankov & Crawford, 1996a,b ; also see Keren, 1991 ;Harvey, 1997 ; and Yates, 1990 , for
reviews), and provides a powerful window into metacognition and cognitive biases.
A comprehensive overview of different calibration indices is available in Schraw’s (2009) review. Since many
of these (e.g., measures of resolution, calibration, etc.) have split-half reliability coefficients lower than .50 (see
Stankov & Crawford, 1996a ), we focus only on two derived scores: (a) The over/underconfidence bias score (or
simply Bias); and (b) The Discrimination score.
Bias
The Bias score has been used extensively in calibration research. It is calculated as the difference between the
average of the confidence ratings over all attempted items and the percentage of items that were answered cor-
rectly. Thus,
Bias5Average Confidence over all items /C0Percentage of correctly solved items
The resulting score indicates an individual’s tendency to judge the accuracy of his/her performance, on aver-
age. Over-confidence is reflected via a positive bias score, and under-confidence by a negative bias score.
Confidence judgments are considered to be more realistic when Bias approaches zero. As a rule of thumb, if Bias
lies within a 610 limit, it is assumed to have little psychological significance and to reflect reasonably good cali-
bration (Stankov, 1999).
Discrimination Score
The Discrimination score is traditionally calculated as the difference between the average of the confidence rat-
ings assigned to correctly solved items and the average of the confidence ratings assigned to incorrect items.
Thus,
Discrimination 5Confidence for correct items /C0Confidence for incorrect items
The obtained score indicates to what degree an individual has discriminated between correct and incorrect
answers. Positive scores indicate that an individual has discriminated appropriately (i.e., greater confidence for
correct rather than incorrect items), with an increasing magnitude indicative of greater discrimination. Negative
scores, although possible, are seldom seen, as they indicate that greater confidence has been assigned to incorrect
rather than correct items.
A ubiquitous finding, known as the Hard/Easy effect, i s the presence of over-confidence for difficult tasks
and either under-confidence or good calibration for e asy tasks. In analyses based on Item Response Theory
(IRT), the Hard/Easy effect has shown an interact ion between ability level and the size of Bias scores
(see Paek, Lee, Stankov & Wilson 2013 ;Stankov, Lee & Paek 2009 ;Stankov, Lee et al., 2012 :Stankov and
Lee, 2014 ).183 CALIBRATION AND THE JUDGMENT OF ACCURACY
II. EMOTIONAL DISPOSITIONS