How the Computer-Adaptive Test Works

CAT successively selects questions so as to maximize the precision of the exam based on what is known about the examinee from previous questions. From the examinee's perspective, the difficulty of the exam seems to tailor itself to their level of ability. For example, if an examinee performs well on an item of intermediate difficulty, he will then be presented with a more difficult question. Or, if he performed poorly, he would be presented with a simpler question. Compared to static multiple choice tests, computer-adaptive tests require fewer test items to arrive at equally accurate scores. (Of course, there is nothing about the CAT methodology that requires the items to be multiple-choice; but just as most exams are multiple-choice, most CAT exams also use this format.)

The basic computer-adaptive testing method is an iterative algorithm with the following steps:

1. The pool of available items is searched for the optimal item, based on the examinee's current ability estimate

2. The chosen item is presented to the examinee, who then answers it correctly or incorrectly

3. The ability estimate is updated, based upon all prior answers

4. Steps 1–3 are repeated until a termination condition is met

Nothing is known about the examinee prior to the administration of the first item, so the algorithm is generally started by selecting an item of medium, or medium-easy, difficulty as the first item.

As a result of adaptive administration, different examinees receive quite different tests. The psychometric technology that allows equitable scores to be computed across different sets of items is item response theory (IRT). IRT is also the preferred methodology for selecting optimal items which are typically selected on the basis of information rather than difficulty, per se.

The TOEFL and the GRE General Test are currently primarily administered as a computer-adaptive test. A related methodology called multistage testing (MST) or CAST is used in the Uniform Certified Public Accountant Examination. MST avoids or reduces some of the disadvantages of CAT.