We conducted a pre-post attitudinal survey measuring “Attitudes toward STEM” (28-items; α > .90) and “Multi-Ethnic Identity” (12-items; α > .90) among 50 middle schoolers. Students skipped items resulting in incomplete scale scores. Given the small sample size and nature of the data, we plan to use case mean substitution to account for missing data before computing a “mean difference” variable between pre- and post-survey scores. The dependent variable will be mean attitude change and the predictor will be exposure to culturally-responsive STEM curriculum (regression).

The literature is inconclusive in identifying clear cutoffs for the maximum number of items that can be missing per scale/measure and will still allow case mean substitution (values ranged from 20-60%). We are hoping to keep as much data as possible but want to select an appropriate approach.

  • Is there a better and/or more robust way to approach this problem?

  • Are there recommendations for peer-reviewed citations which utilize this approach and/or suggest cutoffs?


I would consider case mean substitution inappropriate under any circumstances - unless you've tried everything else and it just makes no difference. Using something simplistic like that (or like taking the median of all respondents on that question or leading out non responders) with even 20% missing sounds highly problematic to me. I doubt you'll find a respectable publication that is based on some proper theoretical or simulation evaluation that would endorse that. Perhaps you'll find something where someone just claims that without having a clue, or based on having done it for one specific example dataset and liking the answer.

Some form of multiple imputation is often a good choice, but if the question responses are on very discrete ordinal scales or are categorical, you may have to look beyond the very basic vanilla version for normally distributed data. There are probably other similarly performing options that might do something reasonable - e.g. k-nearest neighbour imputation.


Your Answer

