research by rodriguez ( 1 ) has asserted mcq items using three response options ( one correct answer with two distractors ) is comparable to , 
 and possibly preferable over , traditional mcq item formats consisting of four response options ( e.g. , one correct answer with three distractors ) , or five response options 
 ( e.g. , one correct answer with four distractors ) .
some medical educators have also adopted the practice of using 3-option responses on mcq exams as a response to the difficulty experienced 
 in generating additional plausible distractors . while it would seem solving the problem of creating a third or fourth plausible distractor could be fairly easily alleviated by tapping the 
 expertise of one s colleagues , advanced students / residents , etc .
( 2 - 4 ) , some remain committed to administering 
 items with 3-option responses . while there is some research ( and continually budding interest from faculty ) supporting the use of items with 3-option responses
, we must consider what effects 
 this might have on guessing strategies and cut score determination decisions so as to avoid any unintended consequences ( 5 ) .
thus , the purpose of this 
 work is to further explore these critically important considerations that have largely gone ignored in the medical education literature to this point . 
 
guessing on multiple - choice examinations ( mcqs ) has long been recognized as a serious threat to score validity .
psychometricians generally have recognized three types of guessing : 1 ) informed guessing ; 2 ) cued guessing ; and 3 ) random guessing ( 6 - 7 ) .
while most educators recognize informed and cued types of guessing behaviors are common on medical school examinations , most educators have largely dismissed the notion that one could attain a desirable score result while relying purely on random guessing .
after all , on the surface this appears to be a very poor strategy for completing an examination . however , let us look more closely at the impact of this particularly unassuming type of guessing behavior . 
  with random guessing , an examinee s odds of correctly answering any given item increase with fewer response options .
for example , for someone guessing purely at random , s / he would have a 1 in 5 ( 20% ) chance of correctly answering an item with 5 response options ; a 1 in 4 ( 25% ) chance of correctly answering an item with 4 response options ; and a 1 in 3 ( 33% ) chance of answering an item with 3 response options .
one may calculate the probability that an examinee will answer a given number of items correctly on any exam ( of any length ) by using the cumulative binomial distribution in excel .
the formula for calculating these probabilities is presented below : 
 
 p = ( 1 - binom.dist ( a , b , c , true ) ) + binom.dist(a,b,c,false ) 
  where , p is the probability of an examinee getting the score of interest ( or higher ) by randomly guessing during a single administration , a is the score of interest ( e.g. , the number of correct responses ) , 
 b is the length of the exam , and c is the probability of getting an item correct when randomly guessing ( e.g. , 1 divided by the number of response options ) . 
  to illustrate why random guessing is a problem that medical educators should take more seriously , especially when cut scores are used , let us consider a hypothetical example .
imagine a medical education scenario in which students complete an exam consisting of 10 items .
the instructor opted to administer only 10 items because the exam is considered particularly rigorous due to challenging items assessing very complex material .
further , the instructor opted to utilize a 3-option format for responses based on literature suggesting 3 options are largely comparable to 4 options when it is difficult to generate an effective third distractor . now , suppose the instructor conducted a standard setting exercise with colleagues and determined the following raw score performance ranges were appropriate and defensible : 0 - 4 = fail ; 5 - 7 = pass ; 8 - 10 = honors .
in such a scenario , we used the cumulative binomial distribution formula presented previously to determine the probability that a student utilizing a random guessing strategy would attain a given number correct ( see table 1 ) .  
probability of attaining a given raw score with 3 response options on a 10-item test this example demonstrates that a student guessing completely at random would have a .2064 ( about a 20.64% ) probability of passing this exam .
pass verdict is much too large , thus the current cut score between a pass / fail decision must be revised . with most education research
it is customary to use a 95% confidence level , meaning education researchers are willing to accept a 5% chance of making an error . in this example
, this would mean we must move the minimum cut score from 5 ( 50% correct in order to attain a pass verdict ) to 7 ( 70% correct in order to attain a pass verdict ) in order to ensure that no examinees would be likely to pass the exam by simply guessing at random .
as stated previously , because this exam is particularly difficult a shift of the cut score from 5 to 7 would likely be indefensible .
this creates quite the dilemma for a medical educator wishing to administer a rigorous assessment while maintaining fair and defensible performance standards . 
 
now , let us look at the same example using 4- and 5-option responses . in this scenario ,
let us assume the instructor was able to generate three and four plausible distractors , respectively , for each of the 10 items .
the probability that a student utilizing a random guessing strategy would attain a given number correct is presented in table 2 .  
probability of attaining a given raw score with 4- and 5- option responses in this scenario , a student guessing completely at random would have approximately a.07 ( or about an 8% ) probability of passing this exam when each item consisted of 4-response options , and about a .03 ( about 3% ) probability of passing when each item consisted of 5- response options . in both instances ,
the probability of passing the exam is much smaller than the exam consisting of 3-response options . interestingly ,
though , the exam consisting of 4-response options still produces a probability of passing that is greater than the typical 5% threshold we would prefer .
if using 4-response options , the instructor would likely still need to revise the cut score in this scenario .
the result would be a move from 5 ( 50% correct in order to attain a pass verdict ) to 6 ( 60% correct in order to attain a pass verdict ) in order to ensure that examinees would be unlikely to pass the exam by simply guessing at random .
although this shift in the cut score seems less egregious than the shift given 3-response options , it still may be problematic and indefensible in such a scenario .
so , what exactly does this mean for medical educators conducting routine classroom assessments ? 
 
while there is research and a continual budding of support from medical educators to support the use of mcq items with 3 response options , it is critical that one remains cognizant of what effect this will have on guessing strategies , and on cut score determination decisions .
simply put , exams consisting of items with fewer response options will increase the likelihood that examinees utilizing a random guessing approach will pass that exam .
the choice of using 3 , 4 or 5 option responses also has significant implications with regard to setting cut scores and various performance standards .
most medical school classroom assessments carry moderate - to - high stakes for students , thus it is imperative that the placement of a cut score is at a position in which it is highly unlikely an examinee could achieve by random guessing . of course , any decision about what constitutes  highly unlikely  is arbitrary , but given the conventions of education research a level of 5% seems reasonable for classroom assessments .
table 3 provides recommendations for cut score thresholds for the minimum performance category ( e.g. , the threshold for a pass / fail decision ) relative to exam length ( from 10 to 100 items , by increments of 5 ) and the number of item response options . 
 
minimum recommended raw cut score placements for items with 3 , 4 , and 5 options a visual scan of table 3 illustrates the impact random guessing may have on minimum performance category threshold decisions .
the table illustrates the number of items that would need to be answered correctly in order to be confident within a 5% error tolerance that the response pattern was not due to random guessing .
specifically , the fewer response options provided , the greater the minimum ( raw ) cut score need be .
for example , an exam consisting of 50 items would require a minimum ( raw ) cut score of 23 if the items consisted of 3 response options , and a minimum ( raw ) cut score of 16 if the items consisted of 5 response options to ensure examinees could not achieve the lowest meaningful performance category by random guessing given a 5% maximum error tolerance .
the difference of requiring correct answers to 7 additional items when fewer options are presented offers a number of additional considerations for medical educators . specifically , what impact might the decision to use items with 3-response options relative to traditional item formats with 4- or 5-response options have on setting fair and defensible standards ? clearly , fewer response options equate to a higher minimum cut score , but what effects might this have on the substantive meaning of that minimum cut score ? what additional unintended consequences might result from using fewer response options ?   practically speaking , mcqs with 3-options are typically easier to write because they require development of fewer plausible distractors than 4- and 5-option mcqs .
educators should recognize that implausible distractors add to test error and therefore are not contributing meaningfully ( in fact , they are doing quite the opposite ) .
item distractors are often written to diagnose common misconceptions ( e.g. , order of operations ) about a topic .
it benefits medical educators when others , such as fellow content experts or residents , review items and their distractors for plausibility . 
 
in addition to distractor plausibility concerns , the amount of time per item can be problematic when trying to assess multiple content or competency domains .
examinees sometimes run out of time and render random guesses in hopes of gaining additional points .
when this occurs , scores are contaminated with error and the resulting measure of student performance will be inaccurate .
proponents for 3-option responses contend examinees will require less time to complete each item , and may lead to more valid score results . 
 
alternatively , modifying exams in which a significant number of students have historically run out of time could give better estimates of difficulty and reliability than previous versions .
however , in reducing the number of distractors that an item has , there is usually a small decrease in each item s difficulty and the exam s overall reliability ( 1 ) .
this translates to a need for more high - quality items per exam to compensate for diminished difficulty and reliability . 
 
it is well - documented that offering a mix of easy , moderate , and difficult items tends to increase reliability estimates ( 8) .
however , medical education classroom assessments are notorious for suffering from low reliability estimates , largely due to over - representation of easy items ( p - values0.75 ) .
the extent to which items are easy due to their complexity , an artifact of student learning , instructional familiarity effects ( 9 ) , or some other factor certainly must be considered as well . as royal and guskey ( 10 ) note , in the context of classroom assessment , when teaching is effective and learning is successful score results should resemble a negatively skewed distribution with few , if any , students receiving low marks .
thus , the importance of reliability as an indicator of quality is often deceiving under these circumstances and becomes a less important indicator of quality . 
 
discrimination coefficients are often helpful for informing instructors of whether or not students performed as expected . to that end , instructors can have some , albeit subjective , estimate of the degree to which the exam functioned as intended when administered to the given sample .
it should be noted that discrimination coefficients simply need to be positive ( e.g. , 0.01 or higher ) , as opposed to the guidelines proposed for norm - referenced examinations ( e.g. , greater than 0.2 , 0.3 , or 0.4 ) which have largely been misunderstood by most medical educators ( 11 ) .
that is , because classroom assessments often yield high scores for examinees and items difficulty estimates tend to indicate most items are  easy  , the discrimination coefficient becomes less important in this context than it would be in a norm - referenced assessment scenario ( e.g. , mcat exam , etc . ) ( 10,12 ) .  
exam length is another important consideration , as having too few items creates range issues and inflates error estimates .
therefore , 
 educators need to ensure there are enough items to adequately measure students abilities , while simultaneously ensuring someone can not pass an exam by utilizing a random guessing strategy . 

research by fisher ( 13 ) indicated exams should consist of a minimum of 25 items , when possible , because measures produced with this many items yield error estimates 
 that generally are stable while reliability estimates can still reach levels exceeding 0.90 . as a general rule , the more items administered the more statistically stable the scores will be ; however , 
 at some point the cognitive load for examinees may become too great or examinees may have insufficient time , so these factors need to be appropriately considered and properly accounted for when determining 
 an appropriate number of items to administer ( 14 ) .   of course , some individuals familiar with psychometric models might argue the solution for modeling guessing is simple : 
 use a 3-parameter logistic ( 3-pl ) item response theory ( irt ) measurement model to account for a pseudo - guessing parameter to adjust students scores . while this approach 
 is certainly a possibility for individuals working in large - scale , medical licensure and certification testing environments , it is entirely inappropriate for classroom 
 assessments as 3pl models require a minimum sample size of 1,000 examinees ( 15 - 17 ) . 

therefore , corrections for guessing using 3-pl irt models is not a viable option for medical education classroom assessments .
the use of rasch measurement models , however , 
 is a possibility for classroom assessments ( 11 ) and may offer insights about guessing ( 18,19 ) .
guessing on multiple - choice examinations ( mcqs ) has long been recognized as a serious threat to score validity .
psychometricians generally have recognized three types of guessing : 1 ) informed guessing ; 2 ) cued guessing ; and 3 ) random guessing ( 6 - 7 ) .
while most educators recognize informed and cued types of guessing behaviors are common on medical school examinations , most educators have largely dismissed the notion that one could attain a desirable score result while relying purely on random guessing .
after all , on the surface this appears to be a very poor strategy for completing an examination . however , let us look more closely at the impact of this particularly unassuming type of guessing behavior . 
  with random guessing , an examinee s odds of correctly answering any given item increase with fewer response options .
for example , for someone guessing purely at random , s / he would have a 1 in 5 ( 20% ) chance of correctly answering an item with 5 response options ; a 1 in 4 ( 25% ) chance of correctly answering an item with 4 response options ; and a 1 in 3 ( 33% ) chance of answering an item with 3 response options .
one may calculate the probability that an examinee will answer a given number of items correctly on any exam ( of any length ) by using the cumulative binomial distribution in excel .
the formula for calculating these probabilities is presented below : 
 
 p = ( 1 - binom.dist ( a , b , c , true ) ) + binom.dist(a,b,c,false ) 
  where , p is the probability of an examinee getting the score of interest ( or higher ) by randomly guessing during a single administration , a is the score of interest ( e.g. , the number of correct responses ) , 
 b is the length of the exam , and c is the probability of getting an item correct when randomly guessing ( e.g. , 1 divided by the number of response options ) . 

to illustrate why random guessing is a problem that medical educators should take more seriously , especially when cut scores are used , let us consider a hypothetical example .
imagine a medical education scenario in which students complete an exam consisting of 10 items .
the instructor opted to administer only 10 items because the exam is considered particularly rigorous due to challenging items assessing very complex material .
further , the instructor opted to utilize a 3-option format for responses based on literature suggesting 3 options are largely comparable to 4 options when it is difficult to generate an effective third distractor . now , suppose the instructor conducted a standard setting exercise with colleagues and determined the following raw score performance ranges were appropriate and defensible : 0 - 4 = fail ; 5 - 7 = pass ; 8 - 10 = honors .
in such a scenario , we used the cumulative binomial distribution formula presented previously to determine the probability that a student utilizing a random guessing strategy would attain a given number correct ( see table 1 ) .  
probability of attaining a given raw score with 3 response options on a 10-item test this example demonstrates that a student guessing completely at random would have a .2064 ( about a 20.64% ) probability of passing this exam .
pass verdict is much too large , thus the current cut score between a pass / fail decision must be revised . with most education research
it is customary to use a 95% confidence level , meaning education researchers are willing to accept a 5% chance of making an error . in this example
, this would mean we must move the minimum cut score from 5 ( 50% correct in order to attain a pass verdict ) to 7 ( 70% correct in order to attain a pass verdict ) in order to ensure that no examinees would be likely to pass the exam by simply guessing at random . as stated previously ,
because this exam is particularly difficult a shift of the cut score from 5 to 7 would likely be indefensible .
this creates quite the dilemma for a medical educator wishing to administer a rigorous assessment while maintaining fair and defensible performance standards . 
 
now , let us look at the same example using 4- and 5-option responses . in this scenario ,
let us assume the instructor was able to generate three and four plausible distractors , respectively , for each of the 10 items .
the probability that a student utilizing a random guessing strategy would attain a given number correct is presented in table 2 .  
probability of attaining a given raw score with 4- and 5- option responses in this scenario , a student guessing completely at random would have approximately a.07 ( or about an 8% ) probability of passing this exam when each item consisted of 4-response options , and about a .03 ( about 3% ) probability of passing when each item consisted of 5- response options . in both instances ,
the probability of passing the exam is much smaller than the exam consisting of 3-response options . interestingly ,
though , the exam consisting of 4-response options still produces a probability of passing that is greater than the typical 5% threshold we would prefer . if using 4-response options , the instructor would likely still need to revise the cut score in this scenario . the result would be a move from 5 ( 50% correct in order to attain a pass verdict ) to 6 ( 60% correct in order to attain a pass verdict ) in order to ensure that examinees would be unlikely to pass the exam by simply guessing at random .
although this shift in the cut score seems less egregious than the shift given 3-response options , it still may be problematic and indefensible in such a scenario .
so , what exactly does this mean for medical educators conducting routine classroom assessments ? 

while there is research and a continual budding of support from medical educators to support the use of mcq items with 3 response options , it is critical that one remains cognizant of what effect this will have on guessing strategies , and on cut score determination decisions .
simply put , exams consisting of items with fewer response options will increase the likelihood that examinees utilizing a random guessing approach will pass that exam .
the choice of using 3 , 4 or 5 option responses also has significant implications with regard to setting cut scores and various performance standards .
most medical school classroom assessments carry moderate - to - high stakes for students , thus it is imperative that the placement of a cut score is at a position in which it is highly unlikely an examinee could achieve by random guessing . of course , any decision about what constitutes  highly unlikely  is arbitrary , but given the conventions of education research a level of 5% seems reasonable for classroom assessments .
table 3 provides recommendations for cut score thresholds for the minimum performance category ( e.g. , the threshold for a pass / fail decision ) relative to exam length ( from 10 to 100 items , by increments of 5 ) and the number of item response options . 
 
minimum recommended raw cut score placements for items with 3 , 4 , and 5 options a visual scan of table 3 illustrates the impact random guessing may have on minimum performance category threshold decisions .
the table illustrates the number of items that would need to be answered correctly in order to be confident within a 5% error tolerance that the response pattern was not due to random guessing .
specifically , the fewer response options provided , the greater the minimum ( raw ) cut score need be .
for example , an exam consisting of 50 items would require a minimum ( raw ) cut score of 23 if the items consisted of 3 response options , and a minimum ( raw ) cut score of 16 if the items consisted of 5 response options to ensure examinees could not achieve the lowest meaningful performance category by random guessing given a 5% maximum error tolerance .
the difference of requiring correct answers to 7 additional items when fewer options are presented offers a number of additional considerations for medical educators . specifically , what impact might the decision to use items with 3-response options relative to traditional item formats with 4- or 5-response options have on setting fair and defensible standards ?
clearly , fewer response options equate to a higher minimum cut score , but what effects might this have on the substantive meaning of that minimum cut score ? what additional unintended consequences might result from using fewer response options ?   practically speaking , mcqs with 3-options are typically easier to write because they require development of fewer plausible distractors than 4- and 5-option mcqs .
educators should recognize that implausible distractors add to test error and therefore are not contributing meaningfully ( in fact , they are doing quite the opposite ) .
item distractors are often written to diagnose common misconceptions ( e.g. , order of operations ) about a topic .
it benefits medical educators when others , such as fellow content experts or residents , review items and their distractors for plausibility . 
 
in addition to distractor plausibility concerns , the amount of time per item can be problematic when trying to assess multiple content or competency domains .
examinees sometimes run out of time and render random guesses in hopes of gaining additional points . when this occurs , scores are contaminated with error and the resulting measure of student performance will be inaccurate .
proponents for 3-option responses contend examinees will require less time to complete each item , and may lead to more valid score results . 
 
alternatively , modifying exams in which a significant number of students have historically run out of time could give better estimates of difficulty and reliability than previous versions .
however , in reducing the number of distractors that an item has , there is usually a small decrease in each item s difficulty and the exam s overall reliability ( 1 ) .
this translates to a need for more high - quality items per exam to compensate for diminished difficulty and reliability . 
 
it is well - documented that offering a mix of easy , moderate , and difficult items tends to increase reliability estimates ( 8) .
however , medical education classroom assessments are notorious for suffering from low reliability estimates , largely due to over - representation of easy items ( p - values0.75 ) .
the extent to which items are easy due to their complexity , an artifact of student learning , instructional familiarity effects ( 9 ) , or some other factor certainly must be considered as well . as royal and guskey ( 10 ) note , in the context of classroom assessment , when teaching is effective and learning is successful score results should resemble a negatively skewed distribution with few , if any , students receiving low marks .
thus , the importance of reliability as an indicator of quality is often deceiving under these circumstances and becomes a less important indicator of quality . 
 
discrimination coefficients are often helpful for informing instructors of whether or not students performed as expected . to that end
, instructors can have some , albeit subjective , estimate of the degree to which the exam functioned as intended when administered to the given sample .
it should be noted that discrimination coefficients simply need to be positive ( e.g. , 0.01 or higher ) , as opposed to the guidelines proposed for norm - referenced examinations ( e.g. , greater than 0.2 , 0.3 , or 0.4 ) which have largely been misunderstood by most medical educators ( 11 ) .
that is , because classroom assessments often yield high scores for examinees and items difficulty estimates tend to indicate most items are  easy  , the discrimination coefficient becomes less important in this context than it would be in a norm - referenced assessment scenario ( e.g. , mcat exam , etc . )
exam length is another important consideration , as having too few items creates range issues and inflates error estimates .
therefore , 
 educators need to ensure there are enough items to adequately measure students abilities , while simultaneously ensuring someone can not pass an exam by utilizing a random guessing strategy . 

research by fisher ( 13 ) indicated exams should consist of a minimum of 25 items , when possible , because measures produced with this many items yield error estimates 
 that generally are stable while reliability estimates can still reach levels exceeding 0.90 . as a general rule , the more items administered the more statistically stable the scores will be ; however , 
 at some point the cognitive load for examinees may become too great or examinees may have insufficient time , so these factors need to be appropriately considered and properly accounted for when determining 
 an appropriate number of items to administer ( 14 ) .   of course , some individuals familiar with psychometric models might argue the solution for modeling guessing is simple : 
 use a 3-parameter logistic ( 3-pl ) item response theory ( irt ) measurement model to account for a pseudo - guessing parameter to adjust students scores . while this approach 
 is certainly a possibility for individuals working in large - scale , medical licensure and certification testing environments , it is entirely inappropriate for classroom 
 assessments as 3pl models require a minimum sample size of 1,000 examinees ( 15 - 17 ) . 

therefore , corrections for guessing using 3-pl irt models is not a viable option for medical education classroom assessments .
the use of rasch measurement models , however , 
 is a possibility for classroom assessments ( 11 ) and may offer insights about guessing ( 18,19 ) .
practically speaking , mcqs with 3-options are typically easier to write because they require development of fewer plausible distractors than 4- and 5-option mcqs .
educators should recognize that implausible distractors add to test error and therefore are not contributing meaningfully ( in fact , they are doing quite the opposite ) . a good distractor targets learners insufficiency in the content or competency domain being assessed .
item distractors are often written to diagnose common misconceptions ( e.g. , order of operations ) about a topic .
it benefits medical educators when others , such as fellow content experts or residents , review items and their distractors for plausibility . 
 
in addition to distractor plausibility concerns , the amount of time per item can be problematic when trying to assess multiple content or competency domains .
examinees sometimes run out of time and render random guesses in hopes of gaining additional points .
when this occurs , scores are contaminated with error and the resulting measure of student performance will be inaccurate .
proponents for 3-option responses contend examinees will require less time to complete each item , and may lead to more valid score results . 
 
alternatively , modifying exams in which a significant number of students have historically run out of time could give better estimates of difficulty and reliability than previous versions .
however , in reducing the number of distractors that an item has , there is usually a small decrease in each item s difficulty and the exam s overall reliability ( 1 ) .
this translates to a need for more high - quality items per exam to compensate for diminished difficulty and reliability . 
 
it is well - documented that offering a mix of easy , moderate , and difficult items tends to increase reliability estimates ( 8) .
however , medical education classroom assessments are notorious for suffering from low reliability estimates , largely due to over - representation of easy items ( p - values0.75 ) .
the extent to which items are easy due to their complexity , an artifact of student learning , instructional familiarity effects ( 9 ) , or some other factor certainly must be considered as well . as royal and guskey ( 10 ) note , in the context of classroom assessment , when teaching is effective and learning is successful score results should resemble a negatively skewed distribution with few , if any , students receiving low marks .
thus , the importance of reliability as an indicator of quality is often deceiving under these circumstances and becomes a less important indicator of quality . 
 
discrimination coefficients are often helpful for informing instructors of whether or not students performed as expected . to that end , instructors can have some , albeit subjective , estimate of the degree to which the exam functioned as intended when administered to the given sample . it should be noted that discrimination coefficients simply need to be positive ( e.g. , 0.01 or higher ) , as opposed to the guidelines proposed for norm - referenced examinations ( e.g. , greater than 0.2 , 0.3 , or 0.4 ) which have largely been misunderstood by most medical educators ( 11 ) .
that is , because classroom assessments often yield high scores for examinees and items difficulty estimates tend to indicate most items are  easy  , the discrimination coefficient becomes less important in this context than it would be in a norm - referenced assessment scenario ( e.g. , mcat exam , etc . )
exam length is another important consideration , as having too few items creates range issues and inflates error estimates .
therefore , 
 educators need to ensure there are enough items to adequately measure students abilities , while simultaneously ensuring someone can not pass an exam by utilizing a random guessing strategy . 

research by fisher ( 13 ) indicated exams should consist of a minimum of 25 items , when possible , because measures produced with this many items yield error estimates 
 that generally are stable while reliability estimates can still reach levels exceeding 0.90 . as a general rule , the more items administered the more statistically stable the scores will be ; however , 
 at some point the cognitive load for examinees may become too great or examinees may have insufficient time , so these factors need to be appropriately considered and properly accounted for when determining 
 an appropriate number of items to administer ( 14 ) .   of course , some individuals familiar with psychometric models might argue the solution for modeling guessing is simple : 
 use a 3-parameter logistic ( 3-pl ) item response theory ( irt ) measurement model to account for a pseudo - guessing parameter to adjust students scores . while this approach 
 is certainly a possibility for individuals working in large - scale , medical licensure and certification testing environments , it is entirely inappropriate for classroom 
 assessments as 3pl models require a minimum sample size of 1,000 examinees ( 15 - 17 ) . 

therefore , corrections for guessing using 3-pl irt models is not a viable option for medical education classroom assessments .
the use of rasch measurement models , however , 
 is a possibility for classroom assessments ( 11 ) and may offer insights about guessing ( 18,19 ) .
while there is some research to support the use of items with 3-option responses , we must remain cognizant of what effect this will have on guessing strategies and cut score determinations . as demonstrated here , the probability of attaining an invalid score result tends to increase in scenarios involving a small number of items .
much more research on items with 3-option responses are needed to better understand what value , if any , 3-option responses truly adds to assessments , and in what contexts potential benefits might be discernible . 

 introduction : research has asserted mcq items using three response options ( one correct answer with two distractors ) is comparable to , and possibly preferable over , traditional mcq item formats consisting of four response options ( e.g. , one correct answer with three distractors ) , or five response options ( e.g. , one correct answer with four distractors ) . 
 some medical educators have also adopted the practice of using 3-option responses on mcq exams as a response to the difficulty experienced in generating additional plausible distractors . to date , however , little work has explored how 3-option responses might impact validity threats stemming from random guessing strategies , and what impact 3-option responses might have on cut - score determinations , particularly in the context of medical education classroom assessments . 
 the purpose of this work is to further explore these critically important considerations that largely have gone ignored in the medical education literature to this point . 
 
 methods : a cumulative binomial distribution formula was used to calculate the probability that an examinee will answer at random a given number of items correctly on any exam ( of any length ) . by way of a demonstration , 
 a variety of scenarios were presented to illustrate how examination length and the number of response options impact examinees chances of passing a given examination , and how subsequent cut - score decisions may be impacted by these factors . 
 
 results : as a general rule , classroom assessments containing fewer items should utilize traditional 4-option or 5-option responses , whereas assessments of greater length are afforded greater flexibility in potentially utilizing 3-option responses . 
 conclusions : more research on items with 3-option responses is needed to better understand what value , if any , 3-option responses truly add to classroom assessments , and in what contexts potential benefits might be discernible . 
 
Introduction
Overview of Guessing
An Illustrative Example
Implications for Medical Educators 


Conclusion