Assessing the measurement model (MM) of self-report scales is crucial to obtain
valid measurements of individuals’ latent psychological constructs. This entails
evaluating the number of measured constructs and determining which construct is
measured by which item. Exploratory factor analysis (EFA) is the most-used
method to evaluate these psychometric properties, where the number of measured
constructs (i.e., factors) is assessed, and, afterward, rotational freedom is
resolved to interpret these factors. This study assessed the effects of an
acquiescence response style (ARS) on EFA for unidimensional and multidimensional
(un)balanced scales. Specifically, we evaluated (a) whether ARS is captured as
an additional factor, (b) the effect of different rotation approaches on the
content and ARS factors recovery, and (c) the effect of extracting the
additional ARS factor on the recovery of factor loadings. ARS was often captured
as an additional factor in balanced scales when it was strong. For these scales,
ignoring extracting this additional ARS factor, or rotating to simple structure
when extracting it, harmed the recovery of the original MM by introducing bias
in loadings and cross-loadings. These issues were avoided by using informed
rotation approaches (i.e., target rotation), where (part of) the rotation target
is specified according to a priori expectations on the MM. Not extracting the
additional ARS factor did not affect the loading recovery in unbalanced scales.
Researchers should consider the potential presence of ARS when assessing the
psychometric properties of balanced scales and use informed rotation approaches
when suspecting that an additional factor is an ARS factor.