(numerical method) and/or verbally. When percentages are used with multiple-choice items, the lowest rating
point (the starting point) depends on the number of alternatives ( k) given with a question. Specifically, the lowest
point is defined by 100/k. For example, in a 4-option multiple-choice question, the probability of answering the
question correctly by chance is 25%. Similarly, in a 2-option multiple-choice question, 50% is the probability of
answering the question correctly by chance. For open-ended questions, or ‘constructed’ answers, the starting
point is 0%.
Confidence scales may employ specified intervals (e.g., 10%, 20%) or be open-ended (‘Assign any level
between 0% and 100%’). To assist with comprehension of a numerical scale, verbal anchors may also be used.
This is especially helpful with younger participants. For instance, the starting point of a scale is marked as
‘Guessing’ or ‘Not Certain’, while the end of the scale typically contains ‘Absolutely Certain’ and/or ‘Sure’
anchors.
Use of verbal anchors on their own is problematic. First, the translation of such cues into numerical values is
arbitrary and rests on the assumption that all participants understand those cues to mean the same level of cer-
tainty. Second, there are no universally accepted verbal anchors to express different levels of certainty. Verbal
expressions vary across different research conditions, making it difficult to draw comparisons. Males and females
may react differently to verbal and numerical scales. It is therefore important to combine numerical and verbal
anchors when assessing online confidence levels and to explain clearly the reasons for using the lowest confi-
dence point as well as the correspondence between different certainty levels and anchors (see examples below).
Allwood, Granhag, & Jonsson (2006, Allwood, Innes-Ker, Homgren, and Fredin; 2008) examined four different
types of confidence scales with children aged 11 /C012 years and 8 /C09 years, incorporating several theories of proba-
bility and different numerical and verbal anchors. In the so-called ‘picture scale’, pictures of ‘frowning’ or
‘smiley’ faces were accompanied by verbal and numerical expressions (e.g., I’m very unsure, just guessing, 20%
up to I’m very sure/Absolutely sure, 100%). In the ‘line’ scale, participants had to make a mark reflective of their
level of certainty on the non-shaded (50% in Figure 7.1 ) area of a line. Figure 7.1 provides an example of the line
scale for a binary (Yes/No) multiple-choice question. For this scale, the area marked is translated into the rele-
vant confidence level (here 80%).
How sure are you that this answer is right?
The authors reported no differences between these scales, suggesting equivalence in the ability to capture con-
fidence levels and their biases. Recent research with younger children (aged 5 /C07 years) used a new child-
friendly numerical and verbal confidence scale accompanied by cartoon-like stimuli.
Using online assessment , participants rate how confident they are that their answer is correct immediately after
performing a cognitive act, such as responding to a test item. This assessment differs from self-report measures,
as well as prospective judgments, where the individual is asked to make a prediction about their performance
prior to the task. Online confidence ratings are averaged over attempted test items to give an overall confidence
score. It appears that this online method /C0probing the actual cognitive act rather than relying on subjective self-
report questionnaire measures /C0is a good way to assess the Confidence trait in an individual (see Kleitman,
2008;Stankov et al., 1999 ;Stankov & Kleitman, 2008 , for reviews).
Sample
Much online confidence assessment has been carried out with undergraduates. Additionally, Stankov and
Crawford (1996) have studied people over the age of 65 years and studies with adolescents and children have
been reported ( Allwood et al., 2006; Buratti, Allwood, & Johanson, 2013; Buratti, Allwood, & Kleitman 2013;
Kleitman et al., 2010; Kleitman & Gibson, 2011; Kleitman et al., 2013; Morony et al., 2013; Roebers, Krebs, &
Roderer 2014; Stankov, Lee et al., 2012 ; Stankov, Morony, & Lee, 2013; Stankov et al., 2008, 2014).
Reliability
There is overwhelming empirical evidence showing pronounced individual differences in confidence ratings
(see Kleitman, 2008 ;Stankov et al., 1999 ;Stankov, 2000 ;Stankov, Lee et al., 2012 ;Stankov, Pallier et al., 2012 ).Absolutely
unsure
50%Absolutely
sure
100%
xFIGURE 7.1 Example of a Line confidence rating scale
(Allwood et al., 2006 ).177 ONLINE PERFORMANCE BASED MEASURES
II. EMOTIONAL DISPOSITIONS