PSYCHOMETRIC EVIDENCE FOR DERIVED SCORES
Bias Scores
Reliability
Internal Consistency
No internal reliability evidence is currently available.
Test/C0Retest
Jonsson and Allwood (2003) reported test /C0retest coefficients for bias scores collected over three time intervals,
each two weeks apart, which correlated .53 (T1&T2), .59 (T2&T3) and .53 (T1&T3) with 79 high school students.
Similarly, Buratti and Allwood (2013) reported significant ( p,.001) overall test /C0retest Goodman-Kruskal
Gamma Correlations of bias scores collected over three time intervals, each one week apart, of .38 (T1&T2), .38
(T2&T3) and .35 (T1&T3) with 30 adults and 61 children (8 to 11 years old).
Parallel Forms/Odd/Even
Stankov and Crawford (1996a,b) reported Parallel Forms and Odd/Even reliabilities for the Bias scores
(labeled ‘overconfidence scores’ in their paper) for five cognitive tests with undergraduate students ( N5114).
The lowest reliability (corrected by Spearman-Brown formula) was .70.
Validity
Convergent/Concurrent
No convergent validity evidence is currently available.
Divergent/Discriminant
No divergent validity evidence is currently available.
Construct/Factor Analytic
The available findings clearly point out that, as with confidence ratings, bias scores from different cognitive
tasks obtained with undergraduate samples ( N.150) converged to define a broad Bias dimension using a prin-
cipal components analysis with promax rotation ( Jackson & Kleitman, 2014; Kleitman, 2008 ). Irrespective of the
nature of the tasks (e.g., Gf, Gc, MDMT) and their difficulty levels, people who were more under- or over-
confident on one type of task tended to be more under- or over-confident on any other type of task relative to
others. This finding strongly supports the importance of taking into account habitual individual differences in
the realism of confidence judgments.
Results and Comments
Although the reliability of Bias scores is adequate and tends to be higher than any other score derived from
the calibration paradigm, we do not recommend the use of these scores in correlational studies. Bias scores, how-
ever, are a very convenient way to depict group differences /C0e.g., overall, females tend to be better calibrated
(have bias scores close to zero) than males.
Discrimination Scores
Reliability
Internal Consistency
No internal reliability evidence is currently available.
Test/C0Retest
Test/C0retest reliability estimates for discrimination have been poor. Buratti and Allwood (2013) reported non-
significant ( p..05) overall test /C0retest Goodman-Kruskal Gamma Correlations of slope scores collected over
three time intervals, each one week apart, of .00 (T1&T2), .11 (T2&T3) and .02 (T1&T3) with 30 adults and 61 chil-
dren (aged 8 to 11 years).184 7. MEASURES OF THE TRAIT OF CONFIDENCE
II. EMOTIONAL DISPOSITIONS